OpenAI’s GPT-6.2 Astra is the second best model in the world on one independent index, and tied for first on another. But when researchers gave it real work to do, it fell well behind Anthropic’s Claude Fable 5.2. The first wave of independent testing since the November 17 launch paints a more complicated picture than OpenAI’s own benchmarks, with Astra winning on cost and 3D reasoning while trailing on coding and on tasks drawn from actual research jobs.

On the Artificial Analysis Intelligence Index, which rolls up nine evaluations across knowledge work, coding, science and general knowledge, GPT-6.2 Astra scores 62 points, second only to Claude Fable 5.2 at 65. Astra jumps five points over GPT-6 Astra, passing Google’s Gemini 4 Carbon at 60 and Anthropic’s Claude Opus 5.5 at 59. GPT-6.2 Sol scores 57, matching GPT-6 Astra at roughly 15 percent of the cost per task. The cost gap is stark: Astra costs about $4 per index task, against $26.54 for Fable 5.2.

Artificial Analysis Intelligence Index scores bar chart.

Chart: GPTOnline, based on Artificial Analysis.

Where Astra Wins, and Where It Does Not

Astra leads on 3D reasoning, topping the AA-3D ranking with 1,318 Elo, ahead of Fable 5.2 at 1,271. It also leads on Terminal-Bench-Science 0.1 and places second behind Fable 5.2 on the physics reasoning test CritPt v2, the factual knowledge benchmark AA-Omniscience and the knowledge benchmark HLE-Diamond, according to Artificial Analysis. But in coding, Astra is flat versus GPT-6 Astra on the Coding Agent Index, and Sol drops three points, a decline Artificial Analysis traces to a 14 point fall on SWE-Atlas-QnA-AA, where Sol more often left out key facts when answering questions about codebases.

Epoch AI, a research organization that tests models on work drawn from its own staff, gives Astra and Fable 5.2 the same point estimate of 174 on its Epoch Capabilities Index, compared with 169 for GPT-6 Astra. Both sit above the previous trend line for reasoning models, Epoch says, and Astra is significantly cheaper.

Real Work Tells a Different Story

The gap opens on Epoch’s Automation Reports, a suite of tasks taken directly from the organization’s own research work, graded by hand against the standard expected of Epoch employees. GPT-6.2 Astra averaged 65 percent, 17 points below Fable 5.2, and no model met the employee standard on any task. On the AI data center analysis task, which asks models to add a data center to Epoch’s database by researching news and regulatory documents, ordering satellite imagery and analyzing images, Fable 5.2 scored 97.5 percent and Opus 5.5 scored 87.5 percent, against 65 percent for Astra. Epoch says Fable 5.2 and Opus 5.5 can now perform that analysis at the level of a contractor, while Astra left out critical information by being overly cautious, covering only one building when its sources clearly pointed to several.

Epoch AI heatmap of model scores on real work tasks.

Average scores across Epoch Automation Reports tasks. Chart: Epoch AI (CC-BY).

Astra did improve over its predecessor in several tasks, reaching performance comparable to Gemini 4 Argon, with a 25 point jump in data center research. But the models still struggle with the same weaknesses. In a task that asked for a research report on chess variants to test out-of-distribution generalization, every model reused variants that already exist, and Astra only swapped piece labels, for example using R for knight instead of rook, which leaves the underlying game unchanged. None of the models iterated effectively on their first attempts, and all of them preferred their own outputs over higher quality human outputs in blind comparisons, except for one case where Gemini 4 Argon chose the human version.

The Caveats, and the User Reaction

Epoch is careful about what its numbers can support. The organization ran each model only once per task, on a small task suite, with manual grading that involves subjective judgment. “Scores are therefore noisy and may shift with more runs,” Epoch writes. “These results should not be read as a general measure of what AI can or cannot do across all work tasks.” The qualitative findings, it says, are more informative than the exact numbers.

Early users have been blunter. VentureBeat reports that the launch drew sharp criticism from several AI power users who tested the models before release, and one widely shared review called Astra “frustrating to use,” saying the model has “no concept of knowledge gaps,” produces outputs that are “very hard to parse,” and “runs in circles on long tasks.”

Why It Matters

The independent results do not overturn OpenAI’s launch claims so much as reframe them. Astra is cheaper and stronger than its predecessor on most of OpenAI’s own benchmarks, and it matches Fable 5.2 on the Epoch Capabilities Index at a fraction of the cost. But the models that look closest on paper are not the closest in practice: on real research work, Fable 5.2 leads by 17 points, and the cheapest model in the comparison is not the one a research team would hand its hardest jobs. For buyers choosing between the two flagships, the question is no longer which model tops an index, but which one survives contact with an actual workday.

Sources