Noometry

Model comparison

Gemini 2.5 Flash vs GPT-5.4

GPT-5.4 is the stronger model overall, scoring 59.4 to 39.3 on the Noometry Index. Gemini 2.5 Flash costs 6.6× less per token, which makes it the better buy when GPT-5.4's lead doesn't matter for your workload.

Last verified . 37 shared benchmarks.

Gemini 2.5 Flash Google

39.3

Rank #170 Confirmed

GPT-5.4 OpenAI

59.4

Rank #16 Confirmed

Summary

  • They share 37 benchmarks with published results for both. Gemini 2.5 Flash scores higher in 0 categories and GPT-5.4 in 10 categories; 10 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where GPT-5.4 leads 61.8 to 18.1.
  • The biggest single-benchmark swing is ARC-AGI-2: 2.5% for Gemini 2.5 Flash and 74% for GPT-5.4.
  • Gemini 2.5 Flash is cheaper at $0.30 / $2.50 per million input/output tokens, against $2.50 / $15 for GPT-5.4.
  • GPT-5.4 accepts more context: 1.05M tokens versus 1.05M.

Side by side

Gemini 2.5 FlashGPT-5.4
ProviderGoogleOpenAI
Noometry Index39.359.4
Released2025-04-172026-03-05
WeightsProprietaryProprietary
Context window1.05M1.05M
Max output66K128K
Input $ / M tokens$0.30$2.50
Output $ / M tokens$2.50$15
Results tracked5468

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.4 leads

Gemini 2.5 Flash: 35.8 (#220), GPT-5.4: 52.6 (#33)

BenchmarkGemini 2.5 FlashGPT-5.4
WeirdML41.9%77.7%
LMArena Coding14241497
ALE-Bench661.881,607
SWE-bench Verified—76.9%
DeepSWE—51.8%
SWE-bench Verified (bash only)28.7%—
Aider Polyglot55.1%—
LMArena WebDev—1465
SciCode—56.6%
GSO—31.4%
MirrorCode—15.6%
AlgoTune—1.85

Agentic & Tool Use GPT-5.4 leads

Gemini 2.5 Flash: 30.8 (#74), GPT-5.4: 46.5 (#13)

BenchmarkGemini 2.5 FlashGPT-5.4
Terminal-Bench17.1%81.8%
Vending-Bench 2548.846,144
APEX-Agents—52.4%
Berkeley Function Calling Leaderboard56.2%—
TheAgentCompany41.1%—
τ²-bench Banking—39.4%
DeepResearch Bench—35.1%
PostTrainBench—19%
BALROG33.5%—
GBAEval—45.1%
LMArena Search—1197
METR Time Horizons—74.3%

Reasoning GPT-5.4 leads

Gemini 2.5 Flash: 18.1 (#286), GPT-5.4: 61.8 (#19)

BenchmarkGemini 2.5 FlashGPT-5.4
ARC-AGI-22.5%74%
Kagi LLM Benchmark56.8%63.8%
ARC-AGI-133.3%93.7%
CritPt1.1%23.4%
EnigmaEval2.7%16%
LMArena Hard Prompts14221485
DTBench76.5%94.4%
LMCA27.5%52%
Epoch Capabilities Index143.03156.81
ForecastBench60.659.5
SimpleBench41.2%—
NYT Connections (extended)—91.3%
Chess Puzzles—44%
Thematic Generalization—80%
EBR-Bench—25.4%
Mystery Game Puzzles—37%

Math GPT-5.4 leads

Gemini 2.5 Flash: 39.9 (#98), GPT-5.4: 73.5 (#19)

BenchmarkGemini 2.5 FlashGPT-5.4
OTIS Mock AIME 2024-202573.1%97.8%
LMArena Math14151488
FrontierMath (Feb 2025 set)4.8%47.6%
FrontierMath Tier 4 (v1)4.2%27.1%
FrontierMath (Tiers 1-3)—78.6%
FrontierMath Tier 4—49%
MathArena Final-Answer Competitions—83.1%
ProofBench—56%
Omni-MATH38.5%—

Knowledge GPT-5.4 leads

Gemini 2.5 Flash: 36.4 (#168), GPT-5.4: 65.3 (#14)

BenchmarkGemini 2.5 FlashGPT-5.4
Humanity's Last Exam12.1%36.2%
Vectara Hallucination Rate7.8%7%
LMArena Expert14261507
GPQA Diamond—93.3%
SimpleQA Verified—45.1%
MMLU-Pro63.9%—
Confabulations16.8%—
GPQA (HELM)39%—

Multimodal GPT-5.4 leads

Gemini 2.5 Flash: 41.8 (#32), GPT-5.4: 43.7 (#20)

BenchmarkGemini 2.5 FlashGPT-5.4
LMArena Vision12531303
GeoBench76%—
VPCT46.2%—
Blueprint-Bench 2—27.1%
Furniture Assembly—37.5%
LMArena Document—1471
SpatialViz-Bench36.9%—

Multilingual GPT-5.4 leads

Gemini 2.5 Flash: 52.3 (#88), GPT-5.4: 56.2 (#23)

BenchmarkGemini 2.5 FlashGPT-5.4
LMArena Non-English14091465
LMArena Chinese14501519
LMArena French14331493
LMArena German14181472
LMArena Japanese14051485
LMArena Korean13851448
LMArena Russian14151480
LMArena Spanish14211454

Instruction Following GPT-5.4 leads

Gemini 2.5 Flash: 75.7 (#54), GPT-5.4: 77.1 (#27)

BenchmarkGemini 2.5 FlashGPT-5.4
LMArena Instruction Following14051469
IFEval89.8%—

Long Context GPT-5.4 leads

Gemini 2.5 Flash: 47.5 (#17), GPT-5.4: 50.3 (#8)

BenchmarkGemini 2.5 FlashGPT-5.4
LMArena Longer Query14191473
Fiction.LiveBench77.8%—
CL-bench—27.9%
CL-bench Life—21.7%

Writing & Preference GPT-5.4 leads

Gemini 2.5 Flash: 53.8 (#157), GPT-5.4: 71.9 (#17)

BenchmarkGemini 2.5 FlashGPT-5.4
LMArena Text14171469
LMArena Creative Writing14001439
EQ-Bench Creative Writing11371840
LMArena Multi-Turn14081482
Short-Story Creative Writing76.5%—
WildBench81.7%—
EQ-Bench 4—1272

Frequently asked questions

Is Gemini 2.5 Flash better than GPT-5.4?

GPT-5.4 is the stronger model overall, scoring 59.4 to 39.3 on the Noometry Index. Gemini 2.5 Flash costs 6.6× less per token, which makes it the better buy when GPT-5.4's lead doesn't matter for your workload.

Which is cheaper, Gemini 2.5 Flash or GPT-5.4?

Gemini 2.5 Flash is cheaper. It lists at $0.30 per million input tokens and $2.50 per million output tokens; GPT-5.4 lists at $2.50 and $15.

Is Gemini 2.5 Flash or GPT-5.4 better for coding?

GPT-5.4 scores higher on coding benchmarks: 52.6 versus 35.8 in the Noometry coding category.

Which has the bigger context window?

GPT-5.4 does, with 1.05M tokens against 1.05M.

How many benchmarks do Gemini 2.5 Flash and GPT-5.4 share?

37 benchmarks have published results for both models. Gemini 2.5 Flash has 54 scored results on Noometry and GPT-5.4 has 68.

Related comparisons

Go deeper