Noometry

Model comparison

Gemini 3 Flash Preview vs GPT-4o

Gemini 3 Flash Preview is the stronger model overall, scoring 52.3 to 28.5 on the Noometry Index.

Last verified . 40 shared benchmarks.

Gemini 3 Flash Preview Google

52.3

Rank #33 Confirmed

GPT-4o OpenAI

28.5

Rank #316 Confirmed

Summary

  • They share 40 benchmarks with published results for both. Gemini 3 Flash Preview scores higher in 10 categories and GPT-4o in 0 categories; 10 gaps are clear of the uncertainty.
  • The widest gap is in math, where Gemini 3 Flash Preview leads 51.4 to 10.2.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 95.6% for Gemini 3 Flash Preview and 6.4% for GPT-4o.
  • Gemini 3 Flash Preview is cheaper at $0.50 / $3 per million input/output tokens, against $2.50 / $10 for GPT-4o.
  • Gemini 3 Flash Preview accepts more context: 1.05M tokens versus 128K.

Side by side

Gemini 3 Flash PreviewGPT-4o
ProviderGoogleOpenAI
Noometry Index52.328.5
Released2025-12-172024-05-13
WeightsProprietaryProprietary
Context window1.05M128K
Max output66K16K
Input $ / M tokens$0.50$2.50
Output $ / M tokens$3$10
Results tracked5972

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 51.5 (#35), GPT-4o: 25.1 (#323)

BenchmarkGemini 3 Flash PreviewGPT-4o
SWE-bench Verified75.4%31%
SWE-bench Verified (bash only)75.8%21.6%
GSO9.8%0%
WeirdML61.6%25.1%
LMArena Coding14601297
Aider Polyglot—45.3%
LMArena WebDev1439—
SWE-bench Multilingual72.7%—
BigCodeBench Instruct—51.1%
LiveBench Coding—51.4%
BigCodeBench Complete—61.1%
CadEval—26%
ALE-Bench1,367—
HumanEval+—87.2%
MBPP+—72.2%

Agentic & Tool Use Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 38.6 (#27), GPT-4o: 20.8 (#133)

BenchmarkGemini 3 Flash PreviewGPT-4o
BALROG48.1%32.3%
LMArena Search11981006
Terminal-Bench64.3%—
GDPval—9.9%
TheAgentCompany—8.6%
τ²-bench Airline82.5%—
τ²-bench Banking27.3%—
τ²-bench Retail76.8%—
τ²-bench Telecom91.2%—
Cybench—12.5%
DeepResearch Bench49.8%—
GDP.pdf10%—
METR Time Horizons—40.8%
Vending-Bench 23,635—

Reasoning Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 49.2 (#35), GPT-4o: 9.4 (#338)

BenchmarkGemini 3 Flash PreviewGPT-4o
ARC-AGI-233.6%0%
SimpleBench61.1%17.8%
ARC-AGI-184.7%4.5%
Chess Puzzles40%13%
LMArena Hard Prompts14651281
DTBench89.1%64.5%
LMCA43.1%16.6%
Epoch Capabilities Index151.83128.97
ForecastBench58.557.7
NYT Connections (extended)83.1%—
CritPt—0%
EnigmaEval—0.8%
LiveBench Reasoning—55.8%
Mystery Game Puzzles26%—
LiveBench Data Analysis—60.9%
LiveBench—55.3%

Math Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 51.4 (#42), GPT-4o: 10.2 (#303)

BenchmarkGemini 3 Flash PreviewGPT-4o
FrontierMath (Tiers 1-3)51.2%0.4%
OTIS Mock AIME 2024-202595.6%6.4%
LMArena Math14721285
FrontierMath (Feb 2025 set)35.6%0.3%
FrontierMath Tier 417.1%—
MathArena Final-Answer Competitions67.6%—
ProofBench15%—
Omni-MATH—29.3%
LiveBench Math—49.5%
MATH Level 5—53.3%
FrontierMath Tier 4 (v1)4.2%—

Knowledge Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 58.8 (#30), GPT-4o: 28.8 (#235)

BenchmarkGemini 3 Flash PreviewGPT-4o
GPQA Diamond89.4%49.2%
SimpleQA Verified66.8%26%
Vectara Hallucination Rate13.5%9.6%
LMArena Expert14611250
Humanity's Last Exam—2.7%
MMLU-Pro—71.3%
Confabulations—15.3%
GPQA (HELM)—52%
MMLU—88.1%

Multimodal Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 45.3 (#15), GPT-4o: 33.9 (#85)

BenchmarkGemini 3 Flash PreviewGPT-4o
LMArena Vision12851137
GeoBench88%71%
VPCT72.6%40%
Video-MME—71.9%
Blueprint-Bench 20%—
LMArena Document1413—
ScienceQA—88.5%

Multilingual Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 55.9 (#26), GPT-4o: 43.4 (#181)

BenchmarkGemini 3 Flash PreviewGPT-4o
LMArena Non-English14581283
LMArena Chinese15101277
LMArena French14781303
LMArena German14971282
LMArena Japanese14881257
LMArena Korean14431234
LMArena Russian14801286
LMArena Spanish14701292

Instruction Following Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 75.8 (#52), GPT-4o: 66.8 (#202)

BenchmarkGemini 3 Flash PreviewGPT-4o
LMArena Instruction Following14371278
LiveBench Instruction Following—68.6%
IFEval—81.7%

Long Context Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 44.4 (#63), GPT-4o: 39.4 (#174)

BenchmarkGemini 3 Flash PreviewGPT-4o
LMArena Longer Query14521289
Fiction.LiveBench—66.7%

Writing & Preference Gemini 3 Flash Preview leads

Gemini 3 Flash Preview: 65.6 (#44), GPT-4o: 52.8 (#160)

BenchmarkGemini 3 Flash PreviewGPT-4o
LMArena Text14661300
LMArena Creative Writing14571291
LMArena Multi-Turn14711302
Short-Story Creative Writing—81.8%
WildBench—82.8%
LiveBench Language—47.6%

Frequently asked questions

Is Gemini 3 Flash Preview better than GPT-4o?

Gemini 3 Flash Preview is the stronger model overall, scoring 52.3 to 28.5 on the Noometry Index.

Which is cheaper, Gemini 3 Flash Preview or GPT-4o?

Gemini 3 Flash Preview is cheaper. It lists at $0.50 per million input tokens and $3 per million output tokens; GPT-4o lists at $2.50 and $10.

Is Gemini 3 Flash Preview or GPT-4o better for coding?

Gemini 3 Flash Preview scores higher on coding benchmarks: 51.5 versus 25.1 in the Noometry coding category.

Which has the bigger context window?

Gemini 3 Flash Preview does, with 1.05M tokens against 128K.

How many benchmarks do Gemini 3 Flash Preview and GPT-4o share?

40 benchmarks have published results for both models. Gemini 3 Flash Preview has 59 scored results on Noometry and GPT-4o has 72.

Related comparisons

Go deeper