AI Agents Score 12% on Real Economic Work: What SemiAnalysis's GDPx Benchmark Shows
Quick verdict
SemiAnalysis released GDPx, a benchmark that scores AI agents on 206 tasks pulled from real, economically valuable work across 11 domains, from chip design and software engineering to finance and biology. The top score was 12%. That was Claude Opus 5.5. Gemini 4 Carbon landed right behind it at 10.9%, and it did so at roughly a quarter of the cost. The headline everyone grabbed was "even the best AI only finishes 12% of real work," and that is fair enough. But if you actually pay for these models, the number next to it matters more. First and second place are about a point apart, the second-place model costs roughly four times less, and no model was good across all 11 domains. Read that way, the benchmark is less a verdict on AI and more a reminder to match the model to the job and keep an eye on the bill.
What the benchmark actually measures
Most benchmarks you see quoted test narrow, academic skills: multiple-choice knowledge, contest math, isolated coding puzzles. GDPx is trying to do something harder. It takes 206 agentic tasks drawn from 11 economically valuable domains and asks whether a model, running as an agent, can actually complete them end to end. These are not trivia. They are the kind of multi-step jobs a competent professional gets paid for.
The scores are low on purpose, because the tasks are hard and graded on finishing, not on partial credit for sounding right. Here is how the top of the board looked:
| Model | GDPx score | Relative cost |
|---|---|---|
| Claude Opus 5.5 | 12% | Highest |
| Gemini 4 Carbon | 10.9% | ~4x lower than Opus |
A 1.1-point lead for Opus, paid for with roughly four times the spend per task. Dylan Patel of SemiAnalysis went as far as publicly apologizing to Google, calling Carbon "spikier" but genuinely strong in some areas. Demis Hassabis welcomed the benchmark. When the people being measured are happy to be measured, the benchmark is usually touching something real.
The scores move once you look closer
The single number hides more than it shows, and the researchers digging into the results found the ranking is less stable than it looks.
- Elie Bakouche pointed out that if you remove the benchmarks affected by safety safeguards, the picture shifts. Carbon stays cheap, and Sol 6.1 comes out cheaper still for the work it does. He also suggested OpenAI's models carry stronger bio safeguards than Anthropic's, which can hold a model back on specific tasks for reasons that have nothing to do with raw ability.
- Within families the spread is wide. Carbon beat Gemini 4 Argon across the board. Sol 6.1 and GPT-6 Astra scored very similarly to each other. Claude Sonnet was strong specifically at software engineering.
- Some of Argon's cybersecurity runs may have been dragged down by a tooling bug rather than the model itself.
"Spiky" is the word that keeps coming up, and it is the most honest summary of the state of AI in late 2026. A model can be near the top on one domain and middling on the next. An average across 11 domains flattens that into a single rank that nobody should treat as destiny.
Why this matters for what you pay
If you only read the 12% headline, the lesson is "AI still can't do most real jobs," and that is true today. But if you are choosing which model to run, the benchmark says something more practical. The best model and the second-best model are separated by about a point, and the cheaper one is cheaper by a lot. Paying the premium for the top-ranked model makes sense when the task sits in its strong domain. For everything else you are often buying a rounding error of quality at four times the price.
This is the whole case for not locking yourself into one provider. When Opus leads on one kind of work, Carbon nearly ties it for a quarter of the cost, Sonnet wins at coding, and Sol is cheapest for a different slice, the smart move is to route each task to the model that fits it rather than send everything through one API and hope it is the right one. We walk through how no single model wins everything in why no single AI model is best at everything, and the head-to-head between the big three in OpenAI vs Anthropic vs Google. Running several models behind one subscription, instead of paying three vendors, is the setup we cover in the best app for running multiple AI models.
There is a cost angle hiding in the "spiky" result too. A benchmark that reports cost per task, not just accuracy, is doing you a favor, because cost per finished task is what actually hits your bill. We looked at that framing in the Briefcase benchmark and cost per task, and at the broader numbers game in AI model benchmarks for 2026.
Video: what the GDPx benchmark shows
A short walkthrough of the GDPx benchmark, what it measures, and why top models still finish only a small fraction of real economic tasks.
FAQ
Does 12% mean AI agents are useless at real work?
No. It means they finish about one in eight of a deliberately hard set of full, multi-step professional tasks with no partial credit. Plenty of real work is easier than a GDPx task, and agents handle a lot of it well. The number pokes a hole in the "AI replaces your job this year" framing. It does not say the tools are no help.
Should I just use the top-ranked model for everything?
Usually not. Opus 5.5 led by about a point while costing roughly four times as much as Gemini 4 Carbon, and different models won different domains. The cheaper option is the better buy for most tasks, with the premium model reserved for the work that sits in its strong area. That is easier to do when you can switch models in one place, which is the point of running multiple models through one app.
Why do the rankings change when you strip out some benchmarks?
Because safety safeguards can stop a model from completing certain tasks for policy reasons, not capability reasons. Remove those tasks and a model that was being penalized for refusing can climb. It is one reason to read any single leaderboard number as a starting point rather than a final answer.
Is GDPx open source?
Not currently. At least one critic noted it is closed and expensive to audit independently, which is a fair caution. Treat the scores as a well-constructed signal from one firm, not as settled fact, until others can reproduce them.
Sources
- @SemiAnalysis_ - GDPx launch, 206 agentic tasks across 11 domains, top score 12%
- @kimmonismus - results breakdown, Opus 5.5 at 12% and Gemini 4 Carbon at 10.9% for ~4x lower cost
- @dylan522p - calling Carbon "spikier" but very strong in some areas
- @eliebakouch - how removing safeguard-affected benchmarks changes the ranking
- @demishassabis - welcoming the GDPx benchmark
Further Reading
Try all the models mentioned in this article
Admix gives you GPT-5, Claude, Gemini, and 300+ AI models in one app. Compare responses side by side. Try it for 7 days for $1.99.
Try Admix for $1.99