Noometry

Model comparison

Gemini 2.5 Pro vs GPT-5.4 mini

Gemini 2.5 Pro and GPT-5.4 mini score almost the same on the Noometry Index (45.0 vs 45.0), so choose on price, context window or the category you care about most.

Last verified . 40 shared benchmarks.

Gemini 2.5 Pro Google

45.0

Rank #75 Confirmed

GPT-5.4 mini OpenAI

45.0

Rank #76 Confirmed

Summary

  • They share 40 benchmarks with published results for both. Gemini 2.5 Pro scores higher in 5 categories and GPT-5.4 mini in 5 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in long context, where Gemini 2.5 Pro leads 59.8 to 43.0.
  • The biggest single-benchmark swing is Kagi LLM Benchmark: 70.3% for Gemini 2.5 Pro and 37.9% for GPT-5.4 mini.
  • GPT-5.4 mini is cheaper at $0.75 / $4.50 per million input/output tokens, against $1.25 / $10 for Gemini 2.5 Pro.
  • Gemini 2.5 Pro accepts more context: 1.05M tokens versus 400K.

Side by side

Gemini 2.5 ProGPT-5.4 mini
ProviderGoogleOpenAI
Noometry Index45.045.0
Released2025-03-252026-03-17
WeightsProprietaryProprietary
Context window1.05M400K
Max output66K128K
Input $ / M tokens$1.25$0.75
Output $ / M tokens$10$4.50
Results tracked7846

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.4 mini leads

Gemini 2.5 Pro: 42.4 (#101), GPT-5.4 mini: 45.2 (#72)

BenchmarkGemini 2.5 ProGPT-5.4 mini
LMArena WebDev12271397
SciCode42.8%49.9%
WeirdML54%60.3%
LMArena Coding14521438
ALE-Bench785.521,189
SWE-bench Verified57.6%—
FrontierCode—27%
SWE-bench Verified (bash only)53.6%—
Aider Polyglot83.1%—
GSO3.9%—
LiveBench Coding85.9%—
CadEval64%—
AlgoTune1.51—

Agentic & Tool Use Too close to call

Gemini 2.5 Pro: 29.2 (#88), GPT-5.4 mini: 29.9 (#81)

BenchmarkGemini 2.5 ProGPT-5.4 mini
DeepResearch Bench42.8%36.3%
Terminal-Bench32.6%—
GDPval23.3%—
Remote Labor Index0.8%—
TheAgentCompany30.3%—
τ²-bench Banking13.7%—
BALROG43.3%—
LMArena Search1142—
METR Time Horizons55.4%—
Vending-Bench 2573.64—

Reasoning GPT-5.4 mini leads

Gemini 2.5 Pro: 28.8 (#99), GPT-5.4 mini: 30.4 (#85)

BenchmarkGemini 2.5 ProGPT-5.4 mini
ARC-AGI-24.9%18.9%
Kagi LLM Benchmark70.3%37.9%
ARC-AGI-141%63.7%
CritPt2%10%
Chess Puzzles20%24%
LMArena Hard Prompts14551424
DTBench82.4%80%
LMCA34.8%40.8%
Epoch Capabilities Index145.32148.84
ForecastBench61.357
SimpleBench62.4%—
NYT Connections (extended)—61.8%
EnigmaEval5.6%—
Thematic Generalization—61.7%
LiveBench Reasoning89.8%—
Mystery Game Puzzles—11%
LiveBench Data Analysis79.9%—
LiveBench82.3%—

Math GPT-5.4 mini leads

Gemini 2.5 Pro: 32.5 (#213), GPT-5.4 mini: 45.5 (#75)

BenchmarkGemini 2.5 ProGPT-5.4 mini
FrontierMath (Tiers 1-3)24.6%51.2%
FrontierMath Tier 40%9.8%
OTIS Mock AIME 2024-202584.7%88.9%
LMArena Math14501419
FrontierMath (Feb 2025 set)14.1%28.3%
FrontierMath Tier 4 (v1)4.2%2.1%
ProofBench—21%
Omni-MATH41.6%—
LiveBench Math90.2%—
MATH Level 595.9%—

Knowledge Gemini 2.5 Pro leads

Gemini 2.5 Pro: 56.0 (#46), GPT-5.4 mini: 51.5 (#67)

BenchmarkGemini 2.5 ProGPT-5.4 mini
GPQA Diamond85.3%86.9%
Vectara Hallucination Rate7%5.5%
LMArena Expert14521435
Humanity's Last Exam21.6%—
SimpleQA Verified—29.4%
MMLU-Pro86.3%—
Confabulations10.6%—
GPQA (HELM)74.9%—

Multimodal Gemini 2.5 Pro leads

Gemini 2.5 Pro: 45.2 (#18), GPT-5.4 mini: 39.7 (#56)

BenchmarkGemini 2.5 ProGPT-5.4 mini
LMArena Vision12631245
GeoBench86%—
VPCT48%—
LMArena Document1421—
SpatialViz-Bench44.7%—

Multilingual Gemini 2.5 Pro leads

Gemini 2.5 Pro: 55.3 (#31), GPT-5.4 mini: 51.9 (#96)

BenchmarkGemini 2.5 ProGPT-5.4 mini
LMArena Non-English14511405
LMArena Chinese15071446
LMArena French14721440
LMArena German14871409
LMArena Japanese14611374
LMArena Korean14341368
LMArena Russian14611417
LMArena Spanish14731405

Instruction Following Too close to call

Gemini 2.5 Pro: 75.0 (#75), GPT-5.4 mini: 74.1 (#102)

BenchmarkGemini 2.5 ProGPT-5.4 mini
LMArena Instruction Following14371405
LiveBench Instruction Following80.6%—
IFEval84%—

Long Context Gemini 2.5 Pro leads

Gemini 2.5 Pro: 59.8 (#5), GPT-5.4 mini: 43.0 (#112)

BenchmarkGemini 2.5 ProGPT-5.4 mini
LMArena Longer Query14491407
Fiction.LiveBench91.7%—

Writing & Preference Too close to call

Gemini 2.5 Pro: 63.7 (#62), GPT-5.4 mini: 64.0 (#58)

BenchmarkGemini 2.5 ProGPT-5.4 mini
LMArena Text14581412
LMArena Creative Writing14541370
EQ-Bench Creative Writing14211665
LMArena Multi-Turn14531429
Short-Story Creative Writing83.8%—
WildBench85.7%—
LiveBench Language67.8%—

Frequently asked questions

Is Gemini 2.5 Pro better than GPT-5.4 mini?

Gemini 2.5 Pro and GPT-5.4 mini score almost the same on the Noometry Index (45.0 vs 45.0), so choose on price, context window or the category you care about most.

Which is cheaper, Gemini 2.5 Pro or GPT-5.4 mini?

GPT-5.4 mini is cheaper. It lists at $0.75 per million input tokens and $4.50 per million output tokens; Gemini 2.5 Pro lists at $1.25 and $10.

Is Gemini 2.5 Pro or GPT-5.4 mini better for coding?

GPT-5.4 mini scores higher on coding benchmarks: 45.2 versus 42.4 in the Noometry coding category.

Which has the bigger context window?

Gemini 2.5 Pro does, with 1.05M tokens against 400K.

How many benchmarks do Gemini 2.5 Pro and GPT-5.4 mini share?

40 benchmarks have published results for both models. Gemini 2.5 Pro has 78 scored results on Noometry and GPT-5.4 mini has 46.

Related comparisons

Go deeper