Gemini 4 Carbon does not hallucinate least. The claim, carried in a November 3, 2026 Tom's Hardware headline that credits the benchmark, fails against the benchmark maker's own table. On Artificial Analysis AA-Omniscience v1.1, published October 31, 2026, Carbon's hallucination rate is 11 percent, nearly three times the 4 percent posted by its predecessor, Gemini 4 Argon. Carbon does lead the test overall, with a Net Score of 54 against 48 for Claude Opus 5.5, but the lead comes from answering more questions right, not from guessing less.
Three numbers, three leaders
AA-Omniscience asks models short factual questions with no context and no tools, then grades each answer as correct, incorrect, partial, or not attempted. It reports three metrics, and each one has a different leader. Net Score, from minus 100 to 100, is 100 times correct minus incorrect, divided by total questions. Accuracy is simply the share of questions answered correctly. Hallucination Rate is incorrect divided by incorrect plus partial plus not attempted: when the model does not have the right answer, how often does it guess anyway?
| Model | Net Score | Accuracy | Hallucination rate |
|---|---|---|---|
| Gemini 4 Carbon (high) | 54 | 58% | 11% |
| Claude Opus 5.5 (max with fallback) | 48 | 66% | Not published in the coverage verified |
| Claude Fable 5.1 (max with fallback) | 46 | 68% | Not published in the coverage verified |
| Gemini 4 Argon (high) | Not published in the coverage verified | 48% | 4% |
| Command A+ | Not published in the coverage verified | Not published in the coverage verified | 14% |
| Muse Spark Open (xhigh) | 26 | Not published in the coverage verified | Not published in the coverage verified |
Anthropic's top models know the most facts: Fable 5.1 answers 68 percent of the questions correctly, the highest accuracy on the table, with Opus 5.5 at 66 percent. Google's Gemini 4 models are the most willing to say they do not know, and Argon is the most disciplined of all, guessing only 4 percent of the time when it lacks the answer. Carbon gets the best balance of the two, which is why it leads Net Score. Muse Spark Open's Net Score of 26 is ByteHerald arithmetic from Artificial Analysis's statement that it trails Carbon by 28 points. The model with the most right answers is not the model with the fewest made-up ones, and Net Score is the compromise between the two.
What Google actually claimed
Google's own launch claim, from the October 28, 2026 announcement of Carbon, which we covered when the model topped the index, is about unsupported claims inside generated answers, which its internal testing says roughly halved compared with Argon. That is a different measure from AA-Omniscience, which tests closed-book recall: short factual questions, no context, no tools. A model can cut unsupported claims in its generated answers while still guessing more often on a closed-book test, because the two tests punish different failures. Google also says Carbon thwarted every attempt in the Gray Swan indirect prompt-injection evaluation. The headline's "least likely to hallucinate" reads as if the two measures were one, and they are not: the wording reflects Carbon's top composite score of 54, not the lowest raw hallucination rate in the benchmark's own figures.
Carbon already leads the aggregate test that contains this one. The Intelligence Index v5 scoreboard we published on November 3 puts Carbon first at 59, one point ahead of Opus 5.5, and AA-Omniscience is one of the nine evaluations inside that index. The new v1.1 table is a refresh of one component, not a new verdict on the whole model, and the refresh itself changed the test: the question set fell from 6,000 to 3,000, saturated and previously public questions were removed, and Artificial Analysis says v1.1 scores should not be compared directly with v1.0 scores.
Why the distinction matters for buyers
Pick by the metric your product needs. For a research or tutoring assistant where recall is the job, accuracy matters most, and Fable 5.1 leads it. For customer-facing answers, legal or health content, or agents that act on facts, a low hallucination rate matters more, and Argon leads that by a wide margin. Give the model permission to abstain: the benchmark prompt explicitly says "I don't know" is better than a wrong answer, and the same instruction belongs in your own system prompt. Watch the effort setting too: several top entries run at max or xhigh effort, which costs more per request, so compare the configuration you would actually deploy.
One benchmark, one grader, is a limit worth stating plainly. Artificial Analysis names GPT-5.6 Luna at medium effort as the grader for v1.1, and the benchmark credibility audit we published in October is the reason we say so: a single grader's judgment is a strong signal, not a final answer. The useful change in v1.1 is not the ranking but the 300-question public set, published under the Apache 2.0 license, which tracks the private scores closely enough to run on your own models. Artificial Analysis scored 86 models on both sets and reported a 0.991 correlation for Net Score. If hallucinations are a risk in your product, run those 300 questions on your shortlist before you pick a model.
The verdict
Carbon is the best balance, Fable knows the most, Argon guesses least. The headline's superlative belongs to Argon, and Argon is the model developers can call today: Carbon had no public API when Artificial Analysis published these results. The honest summary is that Google's newest model answers more questions right and guesses wrong more often than its predecessor, and both facts matter.
By ByteHerald Staff. Artificial Analysis published AA-Omniscience v1.1 on October 31, 2026, with Gemini 4 Carbon leading on Net Score at 54, ahead of Claude Opus 5.5 at 48 and Claude Fable 5.1 at 46, and Gemini 4 Argon posting the lowest hallucination rate at 4 percent against Carbon's 11 percent. Tom's Hardware's reading of the same Artificial Analysis table puts Carbon's accuracy at 58 percent against Argon's 48 percent. Google announced Gemini 4 Carbon on October 28, 2026, and said in internal testing that unsupported claims roughly halved compared with Argon. Tom's Hardware, November 3, 2026, headlined Carbon as the least likely to hallucinate among frontier models, crediting the Artificial Analysis benchmark.
Comments
Post a Comment