Noometry

Model comparison

Gemini 2.5 Pro vs Grok 4.20 (Non-Reasoning)

Grok 4.20 (Non-Reasoning) is the stronger model overall, scoring 48.6 to 45.0 on the Noometry Index.

Last verified . 39 shared benchmarks.

Gemini 2.5 Pro Google

45.0

Rank #75 Confirmed

Grok 4.20 (Non-Reasoning) xAI

48.6

Rank #54 Confirmed

Summary

  • They share 39 benchmarks with published results for both. Gemini 2.5 Pro scores higher in 6 categories and Grok 4.20 (Non-Reasoning) in 4 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Grok 4.20 (Non-Reasoning) leads 52.3 to 28.8.
  • The biggest single-benchmark swing is ARC-AGI-2: 4.9% for Gemini 2.5 Pro and 65.1% for Grok 4.20 (Non-Reasoning).
  • Grok 4.20 (Non-Reasoning) is cheaper at $1.25 / $2.50 per million input/output tokens, against $1.25 / $10 for Gemini 2.5 Pro.
  • Gemini 2.5 Pro accepts more context: 1.05M tokens versus 1M.

Side by side

Gemini 2.5 ProGrok 4.20 (Non-Reasoning)
ProviderGooglexAI
Noometry Index45.048.6
Released2025-03-252026-02-17
WeightsProprietaryProprietary
Context window1.05M1M
Max output66K30K
Input $ / M tokens$1.25$1.25
Output $ / M tokens$10$2.50
Results tracked7846

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

Gemini 2.5 Pro: 42.4 (#101), Grok 4.20 (Non-Reasoning): 42.1 (#112)

BenchmarkGemini 2.5 ProGrok 4.20 (Non-Reasoning)
LMArena WebDev12271375
WeirdML54%52.3%
LMArena Coding14521459
ALE-Bench785.521,150
SWE-bench Verified57.6%—
SWE-bench Verified (bash only)53.6%—
Aider Polyglot83.1%—
SciCode42.8%—
GSO3.9%—
LiveBench Coding85.9%—
CadEval64%—
AlgoTune1.51—

Agentic & Tool Use Grok 4.20 (Non-Reasoning) leads

Gemini 2.5 Pro: 29.2 (#88), Grok 4.20 (Non-Reasoning): 34.4 (#46)

BenchmarkGemini 2.5 ProGrok 4.20 (Non-Reasoning)
Terminal-Bench32.6%57.3%
τ²-bench Banking13.7%18%
LMArena Search11421189
Vending-Bench 2573.644,663
GDPval23.3%—
Remote Labor Index0.8%—
TheAgentCompany30.3%—
DeepResearch Bench42.8%—
BALROG43.3%—
METR Time Horizons55.4%—

Reasoning Grok 4.20 (Non-Reasoning) leads

Gemini 2.5 Pro: 28.8 (#99), Grok 4.20 (Non-Reasoning): 52.3 (#32)

BenchmarkGemini 2.5 ProGrok 4.20 (Non-Reasoning)
ARC-AGI-24.9%65.1%
Kagi LLM Benchmark70.3%75%
ARC-AGI-141%89.5%
Chess Puzzles20%24%
LMArena Hard Prompts14551451
DTBench82.4%90.1%
LMCA34.8%38.7%
Epoch Capabilities Index145.32151.98
ForecastBench61.361.4
SimpleBench62.4%—
NYT Connections (extended)—85.4%
CritPt2%—
EnigmaEval5.6%—
Thematic Generalization—63.8%
LiveBench Reasoning89.8%—
LiveBench Data Analysis79.9%—
LiveBench82.3%—

Math Grok 4.20 (Non-Reasoning) leads

Gemini 2.5 Pro: 32.5 (#213), Grok 4.20 (Non-Reasoning): 48.2 (#65)

BenchmarkGemini 2.5 ProGrok 4.20 (Non-Reasoning)
FrontierMath (Tiers 1-3)24.6%44.9%
FrontierMath Tier 40%17.1%
OTIS Mock AIME 2024-202584.7%92.2%
LMArena Math14501455
ProofBench—14%
Omni-MATH41.6%—
LiveBench Math90.2%—
MATH Level 595.9%—
FrontierMath (Feb 2025 set)14.1%—
FrontierMath Tier 4 (v1)4.2%—

Knowledge Gemini 2.5 Pro leads

Gemini 2.5 Pro: 56.0 (#46), Grok 4.20 (Non-Reasoning): 52.8 (#60)

BenchmarkGemini 2.5 ProGrok 4.20 (Non-Reasoning)
GPQA Diamond85.3%89.3%
LMArena Expert14521439
Humanity's Last Exam21.6%—
SimpleQA Verified—30.2%
MMLU-Pro86.3%—
Confabulations10.6%—
Vectara Hallucination Rate7%—
GPQA (HELM)74.9%—

Multimodal Gemini 2.5 Pro leads

Gemini 2.5 Pro: 45.2 (#18), Grok 4.20 (Non-Reasoning): 33.3 (#98)

BenchmarkGemini 2.5 ProGrok 4.20 (Non-Reasoning)
LMArena Vision12631263
LMArena Document14211416
GeoBench86%—
VPCT48%—
Blueprint-Bench 2—0%
SpatialViz-Bench44.7%—

Multilingual Too close to call

Gemini 2.5 Pro: 55.3 (#31), Grok 4.20 (Non-Reasoning): 54.5 (#40)

BenchmarkGemini 2.5 ProGrok 4.20 (Non-Reasoning)
LMArena Non-English14511441
LMArena Chinese15071481
LMArena French14721476
LMArena German14871465
LMArena Japanese14611449
LMArena Korean14341417
LMArena Russian14611458
LMArena Spanish14731443

Instruction Following Too close to call

Gemini 2.5 Pro: 75.0 (#75), Grok 4.20 (Non-Reasoning): 74.8 (#83)

BenchmarkGemini 2.5 ProGrok 4.20 (Non-Reasoning)
LMArena Instruction Following14371420
LiveBench Instruction Following80.6%—
IFEval84%—

Long Context Gemini 2.5 Pro leads

Gemini 2.5 Pro: 59.8 (#5), Grok 4.20 (Non-Reasoning): 45.5 (#34)

BenchmarkGemini 2.5 ProGrok 4.20 (Non-Reasoning)
LMArena Longer Query14491437
Fiction.LiveBench91.7%—
CL-bench—22.2%
CL-bench Life—11.9%

Writing & Preference Grok 4.20 (Non-Reasoning) leads

Gemini 2.5 Pro: 63.7 (#62), Grok 4.20 (Non-Reasoning): 65.7 (#44)

BenchmarkGemini 2.5 ProGrok 4.20 (Non-Reasoning)
LMArena Text14581451
LMArena Creative Writing14541438
EQ-Bench Creative Writing14211574
LMArena Multi-Turn14531456
Short-Story Creative Writing83.8%—
WildBench85.7%—
LiveBench Language67.8%—

Frequently asked questions

Is Gemini 2.5 Pro better than Grok 4.20 (Non-Reasoning)?

Grok 4.20 (Non-Reasoning) is the stronger model overall, scoring 48.6 to 45.0 on the Noometry Index.

Which is cheaper, Gemini 2.5 Pro or Grok 4.20 (Non-Reasoning)?

Grok 4.20 (Non-Reasoning) is cheaper. It lists at $1.25 per million input tokens and $2.50 per million output tokens; Gemini 2.5 Pro lists at $1.25 and $10.

Is Gemini 2.5 Pro or Grok 4.20 (Non-Reasoning) better for coding?

They score almost the same on coding (42.4 vs 42.1); test both on your own repository before choosing.

Which has the bigger context window?

Gemini 2.5 Pro does, with 1.05M tokens against 1M.

How many benchmarks do Gemini 2.5 Pro and Grok 4.20 (Non-Reasoning) share?

39 benchmarks have published results for both models. Gemini 2.5 Pro has 78 scored results on Noometry and Grok 4.20 (Non-Reasoning) has 46.

Related comparisons

Go deeper