What Happened: OpenAI released GPT-6 Astra on September 3, and the headline benchmark score everyone is sharing came from the software wrapped around the model, not the model on its own. ARC Prize ran the identical model two ways and got 62.7% one way and 98.6% the other. The only thing that changed was whether the system let the model keep its own reasoning between requests.
I have spent two years writing about why AI companions lose the plot. The stock answers are that the model is too small, the context window is too short, or the company nerfed something to save money.
A benchmark result published yesterday points somewhere else entirely.
ARC Prize tested GPT-6 Astra on the same set of games twice. Same model, same reasoning settings, two different harnesses, meaning the scaffolding code that feeds a model its inputs and carries information from one turn to the next.
One run scored 62.7%. The other scored 98.6%, and it cost about nine thousand dollars less to produce.
Nothing about the model changed between those two numbers. What changed was whether it was allowed to remember what it had already worked out. That is not a benchmark curiosity if you use a companion app, because throwing away the character’s accumulated state between messages is how most of them are built.
Below is what Astra can do, what the harness result means, and the specific thing I would now test in any companion app before paying for a year of it.
What Actually Happened
GPT-6 Astra is OpenAI’s new flagship model, released September 3, 2026, and its most-quoted benchmark score came from a custom software wrapper rather than the model alone.
OpenAI announced GPT-6 Astra on September 3, 2026, with a limited preview for trusted partners and a wider rollout to Plus, Pro, Business and Enterprise accounts over the following days.
It carries a 1M token context window, runs at $10 and $50 per million tokens, and is the first OpenAI model to reach the Critical cybersecurity threshold under the company’s Preparedness Framework.
OpenAI president Greg Brockman told reporters that “it’s not unreasonable to feel that we are now in the AGI era.” Take that framing with the salt it deserves.
The benchmark table underneath it is more interesting than the slogan.
| Benchmark | GPT-6 Astra | GPT-5.6 Sol |
|---|---|---|
| OSWorld 2.0 computer use | 72.6% at ~40 min per task | 65.7% at ~75 min per task |
| ScreenSpot-Pro, no tools | 92.7% | 76.9% |
| Agents’ Last Exam | 59.3% | 53.6% |
| AutomationBench | 41.4% | 18.1% |
| ARC-AGI-3 | 99.9% | 7.8% |
| Long-context recall, 512K to 1M tokens | 96.3% | 73.8% |
| Internal hallucination rate, lower is better | 4.2% | 12.2% |
Figures are from OpenAI’s own GPT-6 Astra announcement. The computer-use gains are real and large. Astra completes OSWorld tasks in roughly 47% less time than its predecessor while scoring higher.
The results people are sharing are scrappier than the official table. One tester with early access reported Astra finishing Pokémon FireRed in 18 hours and 12 minutes on a screenshots-only harness with no memory reads and no walkthrough, against 96 hours and 35 minutes for GPT-5.6 Sol and a GPT-5.5 run that never finished after 218 hours.
Those runs are the tester’s own, not OpenAI’s, so treat them as a directional signal rather than a published result.
Then on September 4, ARC Prize published the full breakdown of how the ARC-AGI-3 score was produced, and that is the number worth stopping on.
Why This Is a Bigger Deal Than It Sounds
The software around Astra was worth 36 percentage points on the identical model, more than any reasoning setting.
ARC-AGI-3 drops a model into an interactive environment with no instructions, no stated goal and no rules, then scores how efficiently it learns to operate. When the benchmark launched this year, humans scored 100% and frontier AI models scored 0.51%.
Astra’s published 99.9% is genuine, but it was produced inside OpenAI’s own Provider Adapter. ARC Prize also ran the model through its standard provider-neutral harness.
Here is every reasoning level, side by side, from ARC Prize’s published data as compiled by The New Stack.
| Reasoning effort | ARC Prize standard harness | OpenAI Provider Adapter |
|---|---|---|
| Max | 62.7% for $26,098 | 98.6% for $17,332 |
| XHigh | 59.3% for $37,317 | 98.4% for $18,147 |
| High | 54.8% for $40,705 | 99.9% for $18,817 |
| Medium | 38.6% for $48,090 | 98.4% for $19,285 |
| Low | 17.5% for $38,166 | 98.0% for $21,298 |
| None | 35.2% for $49,791 | 96.7% for $23,457 |
Read the bottom row against the top row, because that comparison is the whole story. Astra with its reasoning effort set to none, inside OpenAI’s adapter, scored 96.7%. Astra at maximum reasoning effort inside the standard harness scored 62.7%.
The dumbest setting in the good wrapper beat the smartest setting in the plain one, by 34 points, and did it for less money.
ARC Prize is specific about what separates the two. The standard harness lets a model carry forward notes it chooses to write down. OpenAI’s adapter preserves the opaque reasoning state between requests and compresses longer conversations, so, in ARC Prize’s words, the model “can resume its own thinking instead of reconstructing it.”
Across the 167 game-and-reasoning pairs both harnesses solved, the adapter runs used 49% fewer tokens and ran roughly 3.66 times faster. Investor Matt Turck summed up the launch benchmarks as “This is wild,” then added the qualifier that matters: “w/ its native harness.”
So the single largest performance variable in the most impressive result of this launch was not intelligence. It was whether the system around the model let it hold onto what it had already figured out.
What This Means for You
Most companion apps rebuild your character from a compressed summary on every message, which is the same architecture that lost by 36 points.
Now put that finding next to how your companion app works.
When you send a message to most character chatbots, the app does not hand the model a live mind that has been sitting there since your last conversation. It assembles a fresh prompt containing the character card, some settings, a compressed summary of your history and the last few messages, then asks a stateless model to perform the character from scratch. Every single turn.
That is the reconstruction path. It is the 62.7% column.
OpenAI described the failure mode plainly in its own release notes, though it was talking about code rather than characters. Writing about long Codex sessions, the company noted that models “have used compaction to summarize work during long sessions” and that “each compaction can leave out details about why a fix failed or how a component behaves.”
With Astra, Codex instead keeps notes across context windows, “preserving accumulated details without repeatedly compressing them into a single summary,” and earlier context windows stay searchable.
Swap the vocabulary and that paragraph is every complaint in every companion subreddit. The lossy summary is why your character remembers that you have a sister but forgets her name.
It is why a personality you spent three months shaping flattens out after a long conversation, and why a favourite character fades without any announced change to the app.
The long-context number matters here too. On a retrieval test that buries eight specific facts inside up to a million tokens of conversation, Astra found 96.3% of them against 73.8% for GPT-5.6 Sol. A model that can reliably find one detail inside a million tokens of history is a different proposition from one that needs your history pre-chewed into a paragraph.
None of this reaches your companion app this week. Astra costs $10 and $50 per million tokens, and companion services run cheaper models on tight per-message margins. But the architecture question is now settled in public with numbers attached, and it gives you something concrete to test.
Before renewing an annual companion subscription, start with the test people get wrong. They ask a question a summary can answer.
Vague: “Do you remember what we talked about last week?”
Specific: “A while back I told you my sister’s name and the city she moved to. What were they?”
The vague version passes on almost any app, because a one-line summary is enough to fake it. The specific version only passes if the app kept the detail itself.
Four things I would check, in order:
- Ask about a specific detail from twenty messages back. Not a theme, a specific. A name, a number, a colour. Summaries preserve themes and destroy specifics, so this tells you whether you are talking to stored history or a compressed abstract of it.
- Check whether the app exposes editable memory. If you can read and correct what the app has stored about you, it is keeping structured state. If memory is invisible, you are trusting a summarizer you cannot inspect.
- Start a fresh session and see what survives. Persistence between sessions is a different feature from persistence within one, and plenty of apps quietly have neither. This is the long-running memory problem that agent builders have been fighting all year.
- Watch for drift after long conversations, not short ones. Character drift shows up when compaction kicks in, which is deep into a session. A ten-message test tells you almost nothing.
If an app fails all four, a model upgrade will not save it. The plumbing is what loses your history, and that is the practical route to keeping a companion engaging. Check it before you blame the model for a memory feature that stopped working.
What Comes Next
Two follow-on stories matter here, and one is unconfirmed while the other is publicly disputed.
Developers have spotted a second model string, gpt-6-astra-aeon, sitting alongside the main one in Codex. The name points at long-running work, and the community has decided it means an always-on persistent agent.
OpenAI has said nothing about it, and a string in an API is not a product, so I would file it as a sighting rather than a roadmap.
The second thread is noisier. The Information reported that Astra uses a technique called recurrent depth, looping its hidden state through the same layers instead of passing through a fixed stack once, which moves part of the reasoning out of readable text.
TechCrunch and Fortune both covered safety researchers who were alarmed by it. OpenAI chief scientist Jakub Pachocki pushed back publicly, arguing the computation-graph depth of current frontier models including Astra sits within a factor of two of GPT-4, and warning against “a race into unmonitorability kicked off by confused reporting.”
The technique appears to be real. The alarm about it is contested. Anyone telling you the model now thinks in secret is reporting one side of an argument that OpenAI’s own chief scientist is publicly having.
What is not contested is the harness result, because ARC Prize published every row of it. The industry just demonstrated, with costs attached, that letting a model keep its own state is worth more than turning its reasoning up. Companion apps have been doing the opposite since the category existed.
Quick Takeaways
- Astra’s headline 99.9% on ARC-AGI-3 came from OpenAI’s Provider Adapter. The same model in ARC Prize’s standard harness scored 62.7%.
- The adapter preserves the model’s reasoning state between requests instead of making it rebuild its thinking, worth 36 points, 49% fewer tokens and roughly 3.66x the speed.
- Companion apps rebuild your character from a compressed summary on every message, which is the same architecture that produced the losing column.
- Test any companion app by asking for a specific detail from twenty messages back, not a general theme, before you pay for a year.
- OSWorld and long-context gains are real and large, but at $10 and $50 per million tokens this generation will not reach budget companion services soon.
