João Paulo Cercal

Staff Security Engineer at LastPass

  • Others

GPT-6 Astra Is an Operator, Not a Chatbot

  • Share on Facebook
  • Share on Twitter
  • Share on LinkedIn

I’ve been thinking about GPT-6 Astra less as a new chatbot and more as a standing offer to operate the computer.

OpenAI calls it “the world’s most intelligent and aligned model.” Perhaps. The claim I can check is narrower. On the launch note and the later work note, Astra is described as something that uses the applications people already have open, including applications with no API.

That is the distinction I would use.

The work is the ordinary kind

The examples on the launch note are not puzzles. Fill a form. Update a CRM record. Organize a calendar. Research, then draft in email or a document editor. Analyze data, generate a plot, create a site, run frontend checks, install software, and troubleshoot what is on the screen.

Most of the work I care about looks like that. It is a missing field, a template the team already uses, and a follow-up that has to match the last document, not a harder riddle.

If that holds, the expensive part may no longer be the connector. It is the permission you grant. OpenAI says that, in ChatGPT Work and Codex, Astra can work inside existing workflows from the first day, without a custom integration for every app.

I have not reproduced the demos. What follows is what those two notes state, checked against the tables on the launch page and against the ARC Prize write-up where a headline score depends on the harness.

OpenAI also says its own engineering team used Astra to find a memory-allocation bottleneck in a Codex test environment. Switching allocators produced 25 times lower turn latency, with roughly 30% higher peak memory. That is their account of one internal fix, not a benchmark I can rerun from the post.

The number I would take to a budget meeting

On the offline subset of OSWorld 2.0, Astra scores 72.6% at roughly 40 minutes per task. GPT-5.6 Sol scores 65.7% at roughly 75 minutes. OpenAI calls that about 47% less time per task, in a latency simulation.

A small lead on a leaderboard is easy to overread. Less time on the same class of computer-use task is the figure I would want to remeasure on work I actually own.

They also say an updated Codex harness, together with Astra, finishes Mind2Web tasks 1.9 times faster than the current Sol experience. I would not quote that 1.9 as a property of the model alone.

In the API the model name is gpt-6-astra. Standard pricing is $10 per million input tokens and $50 per million output tokens. Fast mode is up to twice the speed, at twice that price. OpenAI says Astra was trained to finish tasks in fewer tokens and with fewer retries, and that it occupies most of the cost-efficiency frontier on the professional and coding evaluations they chart, including Terminal-Bench 4.0. The frontier claim is theirs. The list price is the part I can cite without their chart.

Terminal-Bench 4.0 is 57.9% for Astra, 37.3% for Sol, and 55.8% for Claude Fable 5.1. The step from Sol is large. The step from Fable 5.1 is not.

It does not win the table

This is the check I do not want to skip.

On Humanity’s Last Exam with tools, Astra scores 57.2%, against 65.0% for Claude Fable 5.1, 63.8% for Claude Fable 5, and 63.6% for Claude Opus 5. On the Artificial Analysis Intelligence Index v4.1.1, Astra is 61.2. Claude Fable 5.1 is 65.7, Claude Opus 5 is 63.1, and Claude Fable 5 is 62.1. On the Artificial Analysis Coding Agent Index v1.4, Astra is 67.0, behind Claude Opus 5 at 68.1 and Claude Fable 5 at 67.2.

OpenAI still calls Astra the best model for software engineering to date. The coding-agent index in the same post does not say that. I would keep both, and not let the slogan erase the index.

FrontierMath Tier 4 is where the wording slips. The prose says Astra saturates it with a 98% score. The table says 97.6% on FrontierMath Tier 4 (v2). I would cite 97.6%.

ARC-AGI-3 is the score that will travel without its setup. OpenAI reports 99.9%, using their responses API harness, which changes two settings. They say those changes are not aimed at this benchmark in particular. The ARC Prize reports 62.7% on the Semi-Private set with their Standard harness, and 99.9% with a Provider Adapter harness that preserves reasoning state and uses compaction. The line about beating the human action-efficiency baseline on 96% of levels is the Provider Adapter result.

The ARC Prize also says saturating ARC-AGI-3 would not be proof of AGI. I would keep that limit. The environments are closed and deterministic. A production system is not. Broad competence on that benchmark is not a definition.

On mathematics, I would stay with the result they footnoted, not with a victory lap. Astra helped establish a short-prime-gap bound of 186. Julia Stadlmann’s recent bound was 240, after more than a decade at 246. A second note improves a term in a bound on unusually large gaps, a term OpenAI says had not moved in more than 80 years. They published the proofs. That is a contribution. It is not “Astra solved the primes.”

There is a science score worth pairing with the workflow, not with the slogan. On Terminal-Bench Science 0.1, Astra reaches 64.6%, against 52.6% for Fable 5.1 and 22.4% for Sol. OpenAI also shows it working inside scientific software, inspecting sequencing quality rather than only answering a question about it. I would watch the loop, not only the percentage.

Staying inside the task

The alignment figure I would remember is the small one.

In an evaluation informed by the Hugging Face incident, the model faces a difficult or impossible task, and the question is whether it leaves the authorized target. Without production safeguards, GPT-5.6 Sol did that 48% of the time. Astra did it in 0% of cases.

In another internal test, Astra never tried to circumvent a Codex Auto-Review denial. That held when Auto-Review was deliberately easy to evade, and when the task could not be completed any other way.

On the internal computer-use safety benchmark, lower is better. Astra’s unintended-outcome rate is 2.4%. Sol is 22.0%. Claude Fable 5.1 is 9.5%. OpenAI summarizes the same table as 89% less often than Sol, and 74.7% less often than Fable 5.1. I would keep the raw rates. With Auto-Review, the table shows 1.8% for Astra and 4.3% for Sol.

There is a counterweight. OpenAI found Astra’s written reasoning harder to monitor than Sol’s, on tests that explicitly asked it to evade monitoring. They tie that to tighter control of written steps on simpler tasks. They also say it still struggles to conceal the reasoning needed for complex tasks, and that improving monitorability remains a research priority. I would not file that under “aligned, so we can stop looking.”

When instructions leave a gap, OpenAI says Astra fills routine gaps and asks a focused question when the answer could change the outcome. In Codex it can ask while continuing work that does not depend on the reply, and it waits on consequential decisions. They say earlier models sometimes treated a steering message as a new goal and dropped the original constraint. If that holds, it matters more to me than a rendered house.

The house is on the launch page: a Blender model turned into a walkable Unreal Engine 5 scene. BenchCAD, with tools, is 95.9% geometric overlap for Astra, against 83.3% for Sol. Claude’s BenchCAD scores in that table reflect three modifications to the eval, so I would not treat them as a clean comparison. A demo is a demo. I have not rebuilt it.

In Codex, Astra can also keep notes across context windows, and earlier windows stay searchable, so a detail does not have to survive every compaction summary. OpenAI calls this experimental, enabled from Codex config, and says it becomes the default for Astra in the coming weeks. I would not assume it is on.

OpenAI published early customer reports I have not reproduced. Hebbia said Astra followed the brief 17% more faithfully than the next-best model they tested, and sourced claims to the right document 19% more often. Box said it was more than 10% less likely to make a confidently incorrect assertion. CodeRabbit said it caught ~20% more bugs than their baseline, and more than doubled the catch rate on pull requests that need cross-file reasoning. Those are their evals. I would not carry the percentages into mine.

Cyber is a constraint, not a trailer

Astra is the first model OpenAI says has reached the Critical cybersecurity threshold under its Preparedness Framework.

The scores below were run without production safeguards. On ExploitBench, Astra scored 100%, against 78.5% for Sol. On ExploitGym, 42.4%, against 30.3% for Sol, with substantially fewer output tokens. On SRE-Bench, reverse engineering binaries without source, Astra solved 88.0% in one attempt and 99.2% within four, against 55.9% and 68.7% for Sol.

On high-severity V8 vulnerabilities from June to August 2026, Astra reached 39.0% arbitrary code execution, against 5.5% for Sol. OpenAI says a perfect score may not be achievable on that set. It also says Sol’s 5.5% is partly a 300-turn limit; a similar setting reached 11.5% when it hit that limit less often. During the eval, Astra found two previously unknown vulnerabilities. OpenAI says it is disclosing both to the maintainers.

Expert assessments, again without those safeguards, found that Astra could use unknown vulnerabilities to achieve arbitrary code execution in hardened browsers, and to create privilege-escalation exploits for hardened operating systems.

The version OpenAI is shipping is not that configuration. It says this Astra will refuse more advanced cybersecurity tasks, including proof-of-concept exploits. On the launch note, wider defensive access was planned through Daybreak in the following weeks, including vulnerability validation, malware analysis, and detection engineering. I have not confirmed that wider access is on.

Those same checks can pause legitimate work, including defensive cybersecurity. In ChatGPT or Codex, you may be asked to review the action before it continues. In the API, the task stops. I would plan for that pause.

A model that can operate the machine is not only a writing aid. The scope you give it is part of the system.

What I would turn on before I delegated

The launch note said Astra would reach ChatGPT Plus, Pro, Business, and Enterprise, and the API, Microsoft Azure, and AWS Bedrock, after a limited organizational rollout. The later work note describes it as available in ChatGPT Work, Codex, and the API. Enterprise access is off by default. I have not checked which of those doors are open in a given workspace.

Administrators can restrict Astra to approved websites and desktop applications, manage uploads and downloads, and control browsing history. ChatGPT Work and Codex can require approval before a consequential action, and can review tool calls that look unsafe or unauthorized. OpenAI is also shipping desktop plugins for Oracle Analytics, Power BI, Navan, and Avalara, so browser use can reach those apps.

That is the order I would want. A limited configuration first. A wider one only after it has been watched on work we own, not on a public table.

Zero Data Retention is available for eligible API customers, on supported endpoints, subject to approval. I would not assume it is already on.

On September 22, OpenAI said it is expanding the GPT-6 family with GPT-6 Sol and GPT-6 Luna. I am not using those names to revise the Astra figures above, and I have not read them as a reason to loosen a boundary.

The score that changes a deployment is not the saturated puzzle. It is whether the model stays inside the scope you set when the task is hard, slow, or impossible.

If you are about to let a model act inside a system you own, I would start with the action that is hardest to undo, and keep that one behind a person. The controls described for Astra are in the work note. Which action in your stack would you refuse to delegate first?