Noometry

Model comparison

Claude Sonnet 4 vs Gemini 3.5 Flash

Gemini 3.5 Flash is the stronger model overall, scoring 54.1 to 40.6 on the Noometry Index.

Last verified . 34 shared benchmarks.

Claude Sonnet 4 Anthropic

40.6

Rank #139 Confirmed

Gemini 3.5 Flash Google

54.1

Rank #29 Confirmed

Summary

  • They share 34 benchmarks with published results for both. Claude Sonnet 4 scores higher in 1 category and Gemini 3.5 Flash in 9 categories; 10 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Gemini 3.5 Flash leads 62.7 to 23.0.
  • The biggest single-benchmark swing is ARC-AGI-2: 5.9% for Claude Sonnet 4 and 72.1% for Gemini 3.5 Flash.
  • Gemini 3.5 Flash is cheaper at $1.50 / $9 per million input/output tokens, against $3 / $15 for Claude Sonnet 4.
  • Gemini 3.5 Flash accepts more context: 1.05M tokens versus 200K.

Side by side

Claude Sonnet 4Gemini 3.5 Flash
ProviderAnthropicGoogle
Noometry Index40.654.1
Released2025-05-222026-05-19
WeightsProprietaryProprietary
Context window200K1.05M
Max output64K66K
Input $ / M tokens$3$1.50
Output $ / M tokens$15$9
Results tracked5854

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 3.5 Flash leads

Claude Sonnet 4: 44.0 (#70), Gemini 3.5 Flash: 49.1 (#46)

BenchmarkClaude Sonnet 4Gemini 3.5 Flash
SciCode40%53.1%
WeirdML46.1%62.6%
LMArena Coding14141492
ALE-Bench655.35911.02
SWE-bench Verified—79.3%
DeepSWE—37.4%
SWE-bench Verified (bash only)64.9%—
Aider Polyglot61.3%—
LMArena WebDev—1499
GSO4.9%—

Agentic & Tool Use Claude Sonnet 4 leads

Claude Sonnet 4: 38.3 (#29), Gemini 3.5 Flash: 24.8 (#107)

BenchmarkClaude Sonnet 4Gemini 3.5 Flash
APEX-Agents—27.5%
TheAgentCompany33.1%—
Cybench35%—
DeepResearch Bench46.6%—
OSWorld43.9%—
GBAEval—6.7%
GDP.pdf—14%
METR Time Horizons62%—
Vending-Bench 2—5,396

Reasoning Gemini 3.5 Flash leads

Claude Sonnet 4: 23.0 (#169), Gemini 3.5 Flash: 62.7 (#16)

BenchmarkClaude Sonnet 4Gemini 3.5 Flash
ARC-AGI-25.9%72.1%
SimpleBench45.5%76.7%
ARC-AGI-140%92.5%
CritPt0.3%13.1%
EnigmaEval3.1%25.4%
LMArena Hard Prompts13721488
DTBench77.1%94.7%
LMCA29%47.1%
Epoch Capabilities Index141.69154.55
ForecastBench60.259
Kagi LLM Benchmark73%—
NYT Connections (extended)—92.6%
Chess Puzzles—50%
EBR-Bench—4.8%
Mystery Game Puzzles—32%
Surface Evolver Bench—58.1%

Math Gemini 3.5 Flash leads

Claude Sonnet 4: 41.5 (#71), Gemini 3.5 Flash: 60.5 (#30)

BenchmarkClaude Sonnet 4Gemini 3.5 Flash
OTIS Mock AIME 2024-202571.1%95.6%
LMArena Math13751502
FrontierMath (Feb 2025 set)4.1%39%
FrontierMath Tier 4 (v1)0%14.6%
FrontierMath (Tiers 1-3)—62.8%
FrontierMath Tier 4—26.8%
MathArena Final-Answer Competitions—76.3%
ProofBench—31%
Omni-MATH60.2%—
MATH Level 584.4%—

Knowledge Gemini 3.5 Flash leads

Claude Sonnet 4: 41.6 (#99), Gemini 3.5 Flash: 66.3 (#10)

BenchmarkClaude Sonnet 4Gemini 3.5 Flash
GPQA Diamond79.2%92.8%
LMArena Expert13711493
Humanity's Last Exam7.8%—
SimpleQA Verified—66.2%
MMLU-Pro84.3%—
Confabulations13.2%—
Vectara Hallucination Rate10.3%—
GPQA (HELM)70.6%—

Multimodal Gemini 3.5 Flash leads

Claude Sonnet 4: 25.6 (#116), Gemini 3.5 Flash: 45.7 (#13)

BenchmarkClaude Sonnet 4Gemini 3.5 Flash
LMArena Vision11911311
GeoBench37%—
VPCT34%—
Blueprint-Bench 2—33.6%
LMArena Document—1463
MindCube44.8%—

Multilingual Gemini 3.5 Flash leads

Claude Sonnet 4: 46.9 (#151), Gemini 3.5 Flash: 57.1 (#12)

BenchmarkClaude Sonnet 4Gemini 3.5 Flash
LMArena Non-English13331475
LMArena Chinese13511525
LMArena French13641490
LMArena German13311500
LMArena Japanese13011481
LMArena Korean12931453
LMArena Russian13551491
LMArena Spanish13561478

Instruction Following Gemini 3.5 Flash leads

Claude Sonnet 4: 71.8 (#141), Gemini 3.5 Flash: 77.2 (#29)

BenchmarkClaude Sonnet 4Gemini 3.5 Flash
LMArena Instruction Following13751466
IFEval84%—

Long Context Gemini 3.5 Flash leads

Claude Sonnet 4: 33.6 (#254), Gemini 3.5 Flash: 45.3 (#39)

BenchmarkClaude Sonnet 4Gemini 3.5 Flash
LMArena Longer Query13981481
Fiction.LiveBench46.9%—

Writing & Preference Gemini 3.5 Flash leads

Claude Sonnet 4: 57.3 (#127), Gemini 3.5 Flash: 65.5 (#46)

BenchmarkClaude Sonnet 4Gemini 3.5 Flash
LMArena Text13511481
LMArena Creative Writing13451467
LMArena Multi-Turn13761482
Short-Story Creative Writing81.4%—
EQ-Bench Creative Writing1483—
WildBench83.8%—
EQ-Bench 4—1087

Frequently asked questions

Is Claude Sonnet 4 better than Gemini 3.5 Flash?

Gemini 3.5 Flash is the stronger model overall, scoring 54.1 to 40.6 on the Noometry Index.

Which is cheaper, Claude Sonnet 4 or Gemini 3.5 Flash?

Gemini 3.5 Flash is cheaper. It lists at $1.50 per million input tokens and $9 per million output tokens; Claude Sonnet 4 lists at $3 and $15.

Is Claude Sonnet 4 or Gemini 3.5 Flash better for coding?

Gemini 3.5 Flash scores higher on coding benchmarks: 49.1 versus 44.0 in the Noometry coding category.

Which has the bigger context window?

Gemini 3.5 Flash does, with 1.05M tokens against 200K.

How many benchmarks do Claude Sonnet 4 and Gemini 3.5 Flash share?

34 benchmarks have published results for both models. Claude Sonnet 4 has 58 scored results on Noometry and Gemini 3.5 Flash has 54.

Related comparisons

Go deeper