Gemini 4 Carbon Takes the Top Spot on the Intelligence Index, Then Hands It Caveats
Quick verdict
Google shipped a second frontier model in two weeks, and this one took the lead. Gemini 4 Carbon sits at number one on Artificial Analysis's rebuilt Intelligence Index v5 with a score of 59, a point ahead of Claude Opus 5.5, and it does the same index tasks for less money. It is pitched as a cybersecurity and long-horizon model, with an 88.2% on SEC-bench Pro and a 1M-token context. Two things keep it from being a clean win. Its hallucination rate jumped from Argon's 4% to 11%, which is an awkward number for a model whose headline job is finding and patching security bugs. And like Argon, almost nobody can run it: access opens with vetted cyber defenders through the Fairwind Program, and everyone else waits. So Google is genuinely back on top of the general-intelligence chart, on a launch you mostly have to take on trust.
What actually shipped
Carbon lands two weeks after Gemini 4 Argon and aims at the same three jobs: long-horizon coding, enterprise knowledge work, and cyber defense. The specs Google led with:
- Cyber results: 88.2% on SEC-bench Pro for finding vulnerabilities, 54.8% on ExploitGym for exploitation, and 71.0% on CWE-bench for patching. These are the numbers the whole launch is built around.
- Pricing: Phil Schmid quotes $2/$10 per million input/output tokens, with an asterisk. Artificial Analysis lists the standard price at $4/$20 with a 50% launch discount that has no announced end date, and cached input at $0.20 per million.
- Specs: a 1M-token context window and a 1M max output. It takes text, image, video, and speech in, and returns text.
- Availability: trusted cyber defenders first, through the Fairwind Program, with wider developer access promised "as soon as possible" and no date attached.
Google also put real weight on behavior, not just capability. On Terminal-Bench 4.0, it says successful cheating fell to 0.0% and cheating attempts to 0.4%, unsupported claims dropped about 2x against Argon, overreach fell 26%, and under-service fell 29%. For a model going to security teams first, "does what you asked and does not quietly fake it" is arguably the pitch that matters more than any single benchmark.
Where it lands against Astra and Opus 5.5
The independent read is the one worth your time, because three outside labs ran Carbon and they mostly agree: top-tier general model, real cost advantage, clear soft spots on pure coding.
Artificial Analysis put Carbon first on Intelligence Index v5 at 59, leading four of the nine evals in the index (HLE-Diamond, TB-Science, AA-Omniscience, and AutomationBench). The cost line is the genuinely interesting part:
| Model | Intelligence Index v5 | Cost per index task | Output tokens/task |
|---|---|---|---|
| Gemini 4 Carbon | 59 | $7.76 | 123K |
| Claude Opus 5.5 | 58 | $10.04 | ~173K |
| GPT-6 Astra | 57 | $6.25 | — |
| GPT-6.1 Sol | 55 | — | — |
Carbon uses 29% fewer output tokens per task than Opus 5.5 and comes in cheaper per task, which is the opposite of the Argon story, where Google won on intelligence but burned tokens doing it. So this is a real efficiency gain, not just a discount. Astra is still cheaper, though, and the gap narrows once Carbon's intro pricing lapses.
The other labs filled in the picture. On Arena, Carbon is first in Text Arena at 1528, second in Agent Arena at a $1.80 median cost per task, and fourth in Code Arena's WebDev track. Vals ranked it second on its index but first on both the Finance Index and the Legal Research Benchmark, where it produces 64% more output than Argon. Andon Labs put it first on Vending-Bench 2 and first on Blueprint-Bench 2, where it tied the human baseline for 3D spatial reasoning.
Where it is not first is the tell. On FrontierSWE, Carbon scores 61.2%, behind GPT-6 Astra (65.5), Opus 5.5 (62.3), and even Sonnet 5.5 (61.9). On the Coding Agent Index it hits 59 using Google's own Antigravity CLI, against 62 for Claude Code running Opus 5.5. One reviewer flagged that the coding numbers were all measured inside Antigravity, so how Carbon does in other harnesses is untested. For raw software engineering, the frontier lead is still Anthropic's and OpenAI's.
The hallucination problem is the real story
Carbon leads AA-Omniscience with a score of 54, and its hallucination rate is still the second lowest anyone has measured, well under the 50% that both Opus 5.5 and Astra post. But it rose from Argon's 4% to 11%, nearly triple, in one release. That tradeoff is defensible on most work: a model that answers more and abstains less will invent more, and 11% beats the field. It is a harder sell when the product you are shipping is aimed at cyber defenders, and some reviewers said exactly that. A security tool that confidently names a vulnerability that is not there, or patches the wrong line, costs more than one that says "I am not sure." Google gated the launch behind safety work for a reason, and this is the number to watch when outside teams finally run it on their own code.
There is a second asterisk around the headline ranking itself. Artificial Analysis rebuilt the Intelligence Index to v5 the same week, and 60% of the new index now runs on private test sets. That is good for stopping benchmark contamination, but it also means the chart that crowns Carbon is harder for anyone to reproduce, and the timing of a methodology reset that happens to coincide with a Google launch drew the obvious skepticism. The honest read: Carbon is a genuine top-tier model, and the exact ranking is softer than a single number makes it look. Scaling01, who tracks this closely, still thinks Google is roughly two months behind the next OpenAI and Anthropic releases.
What it means if you pay for AI
Nothing you can act on today, because Carbon is locked to cyber defenders, same as Argon was. But the direction is clear. The three-way race at the top is not just real, it now rotates: Argon traded blows with Astra two weeks ago, and Carbon just edged past Opus 5.5. No single model holds the crown long enough to build a subscription around, and each one wins on a different axis, Carbon on cost-per-task and hallucination, Astra and Opus on pure coding, others on knowledge work. For how the three labs stack up overall, see our OpenAI vs Anthropic vs Google breakdown, and our Gemini 4 Argon launch piece for the model Carbon just replaced.
The lesson for anyone budgeting around this is the same as it was for Argon: the sticker price keeps falling, but the real cost is tokens-per-task times the price, and the best model flips every couple of weeks. Carbon doing a task in 29% fewer tokens than Opus is the kind of detail a flat monthly plan can never pass on to you. The sane setup is to route each job to whatever model is currently cheapest for the quality you need, instead of committing to one lab's pricing, which is the entire point of an AI aggregator. When Carbon opens to developers, the question will not be whether it is good. It is whether its cost edge survives the intro discount, and whether an 11% hallucination rate is low enough for the work you would actually hand it.
Video: Gemini 4 Carbon explained
A walkthrough of what Carbon claims, where it leads, and where the benchmark card oversells, for anyone who wants more than the launch slide.
FAQ
Can I use Gemini 4 Carbon right now?
Not unless you are a vetted cyber defender in Google's Fairwind Program. Google says wider developer access is coming "as soon as possible" but has not given a date, the same gated rollout it used for Argon.
How much does Gemini 4 Carbon cost?
Phil Schmid quoted $2/$10 per million input/output tokens. Artificial Analysis lists standard pricing at $4/$20 with a 50% launch discount that has no announced end date, plus $0.20 per million for cached input. Carbon also uses about 29% fewer output tokens per task than Claude Opus 5.5, so its real cost-per-task is lower, roughly $7.76 against Opus's $10.04 on the Intelligence Index.
Is Gemini 4 Carbon better than Opus 5.5 or GPT-6 Astra?
On general intelligence, yes by a hair: it is first on Artificial Analysis's Intelligence Index v5 at 59, a point above Opus 5.5 and two above Astra. On pure software engineering it trails both, scoring 61.2% on FrontierSWE against Astra's 65.5 and Opus's 62.3. It wins on cost-per-task and hallucination rate. See our OpenAI vs Anthropic vs Google guide for the full comparison.
Why does the hallucination rate matter so much here?
Carbon's hallucination rate rose from Argon's 4% to 11%. That is still the second lowest measured and far below the 50% that Opus 5.5 and Astra post, but it tripled in one release, and the model's headline use case is cybersecurity, where a confidently wrong answer about a vulnerability can do real damage. It is the number to watch once outside teams can test it.
Sources
- Sundar Pichai - Gemini 4 Carbon announcement and cyber benchmarks
- Google DeepMind - introducing Gemini 4 Carbon
- Google AI - Fairwind Program rollout for cyber defenders
- Philipp Schmid - $2/$10 intro pricing
- Artificial Analysis - Carbon #1 on Intelligence Index v5, cost and hallucination
- Logan Kilpatrick - Terminal-Bench cheating and safety deltas
- Vals - Carbon #1 on the Finance Index and Legal Research Benchmark
- Arena - Carbon #1 in Text Arena at 1528
- Proximal - Carbon 61.2% on FrontierSWE, behind Astra and Opus
- Andon Labs - Carbon #1 on Vending-Bench 2
- The Rundown - Intelligence Index v5 now 60% private tests
- scaling01 - Google still roughly two months behind OpenAI and Anthropic
- r/singularity - Gemini 4 Carbon benchmark discussion
Further reading
Try all the models mentioned in this article
Admix gives you GPT-5, Claude, Gemini, and 350+ AI models in one app. Compare responses side by side. Try it for 7 days for $1.99.
Try Admix for $1.99