Noometry

Model comparison

Gemini 3.1 Pro Preview vs o3

Gemini 3.1 Pro Preview is the stronger model overall, scoring 56.7 to 47.5 on the Noometry Index.

Last verified . 45 shared benchmarks.

Gemini 3.1 Pro Preview Google

56.7

Rank #23 Confirmed

o3 OpenAI

47.5

Rank #61 Confirmed

Summary

  • They share 45 benchmarks with published results for both. Gemini 3.1 Pro Preview scores higher in 7 categories and o3 in 3 categories; 10 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Gemini 3.1 Pro Preview leads 71.7 to 32.0.
  • The biggest single-benchmark swing is ARC-AGI-2: 77.1% for Gemini 3.1 Pro Preview and 6.5% for o3.
  • o3 is cheaper at $2 / $8 per million input/output tokens, against $2 / $12 for Gemini 3.1 Pro Preview.
  • Gemini 3.1 Pro Preview accepts more context: 1.05M tokens versus 200K.

Side by side

Gemini 3.1 Pro Previewo3
ProviderGoogleOpenAI
Noometry Index56.747.5
Released2026-02-192025-04-16
WeightsProprietaryProprietary
Context window1.05M200K
Max output66K100K
Input $ / M tokens$2$2
Output $ / M tokens$12$8
Results tracked7163

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3 leads

Gemini 3.1 Pro Preview: 42.5 (#99), o3: 46.8 (#64)

BenchmarkGemini 3.1 Pro Previewo3
SWE-bench Verified75.6%62.3%
GSO22.6%8.8%
WeirdML72.1%52.4%
LMArena Coding14841408
ALE-Bench1,161933.55
DeepSWE11.7%—
SWE-bench Verified (bash only)—58.4%
Aider Polyglot—81.3%
LMArena WebDev1447—
SciCode58.9%—
MirrorCode8.9%—
CadEval—74%
AlgoTune2.02—

Agentic & Tool Use Gemini 3.1 Pro Preview leads

Gemini 3.1 Pro Preview: 37.7 (#34), o3: 34.5 (#44)

BenchmarkGemini 3.1 Pro Previewo3
DeepResearch Bench47.8%45.2%
LMArena Search12111144
METR Time Horizons77%65.4%
Terminal-Bench80.2%—
APEX-Agents35.3%—
Berkeley Function Calling Leaderboard—63%
GDPval—30.8%
τ²-bench Banking26%—
OSWorld—23%
PostTrainBench22%—
BALROG57%—
ExploitBench26.1%—
GBAEval0.8%—
GDP.pdf17%—
Vending-Bench 23,774—

Reasoning Gemini 3.1 Pro Preview leads

Gemini 3.1 Pro Preview: 71.7 (#12), o3: 32.0 (#78)

BenchmarkGemini 3.1 Pro Previewo3
ARC-AGI-277.1%6.5%
SimpleBench79.6%53.1%
ARC-AGI-198%60.8%
CritPt17.7%1.4%
Chess Puzzles55%38%
EnigmaEval36.8%13.1%
LMArena Hard Prompts14851402
Mystery Game Puzzles34%29%
DTBench97.1%84.8%
LMCA53.8%39.7%
Epoch Capabilities Index154.77146.86
ForecastBench5962.5
Kagi LLM Benchmark—67.6%
NYT Connections (extended)97.4%—
Thematic Generalization79.4%—
EBR-Bench14.3%—

Math Gemini 3.1 Pro Preview leads

Gemini 3.1 Pro Preview: 62.1 (#34), o3: 50.2 (#58)

BenchmarkGemini 3.1 Pro Previewo3
FrontierMath (Tiers 1-3)59.6%33.3%
OTIS Mock AIME 2024-202595.6%84.4%
LMArena Math14851426
FrontierMath (Feb 2025 set)36.9%18.7%
FrontierMath Tier 4 (v1)16.7%2.1%
FrontierMath Tier 426.8%—
MathArena Final-Answer Competitions86.5%—
ProofBench26%—
Omni-MATH—71.4%
MATH Level 5—97.8%

Knowledge Gemini 3.1 Pro Preview leads

Gemini 3.1 Pro Preview: 71.8 (#3), o3: 54.6 (#52)

BenchmarkGemini 3.1 Pro Previewo3
GPQA Diamond94.4%81.8%
Humanity's Last Exam46.4%20.3%
SimpleQA Verified73.5%49.4%
LMArena Expert14851402
MMLU-Pro—85.9%
Confabulations—14.4%
Vectara Hallucination Rate10.4%—
GPQA (HELM)—75.3%

Multimodal o3 leads

Gemini 3.1 Pro Preview: 37.9 (#69), o3: 41.4 (#36)

BenchmarkGemini 3.1 Pro Previewo3
LMArena Vision12961214
GeoBench—74%
VPCT—52%
Blueprint-Bench 226.5%—
Furniture Assembly26.7%—
LMArena Document1444—

Multilingual Gemini 3.1 Pro Preview leads

Gemini 3.1 Pro Preview: 57.0 (#12), o3: 51.7 (#105)

BenchmarkGemini 3.1 Pro Previewo3
LMArena Non-English14771401
LMArena Chinese15291437
LMArena French14871430
LMArena German14911420
LMArena Japanese14931403
LMArena Korean14551370
LMArena Russian14981406
LMArena Spanish14791395

Instruction Following Gemini 3.1 Pro Preview leads

Gemini 3.1 Pro Preview: 77.0 (#32), o3: 72.8 (#127)

BenchmarkGemini 3.1 Pro Previewo3
LMArena Instruction Following14661368
IFEval—86.9%

Long Context o3 leads

Gemini 3.1 Pro Preview: 47.4 (#18), o3: 53.3 (#6)

BenchmarkGemini 3.1 Pro Previewo3
CL-bench20.8%17.8%
LMArena Longer Query14831372
Fiction.LiveBench—88.9%
CL-bench Life16.9%—

Writing & Preference Gemini 3.1 Pro Preview leads

Gemini 3.1 Pro Preview: 66.1 (#37), o3: 63.5 (#64)

BenchmarkGemini 3.1 Pro Previewo3
LMArena Text14811410
LMArena Creative Writing14821359
EQ-Bench Creative Writing14911676
LMArena Multi-Turn14881405
Short-Story Creative Writing—83.9%
WildBench—86.1%
EQ-Bench 41142—

Frequently asked questions

Is Gemini 3.1 Pro Preview better than o3?

Gemini 3.1 Pro Preview is the stronger model overall, scoring 56.7 to 47.5 on the Noometry Index.

Which is cheaper, Gemini 3.1 Pro Preview or o3?

o3 is cheaper. It lists at $2 per million input tokens and $8 per million output tokens; Gemini 3.1 Pro Preview lists at $2 and $12.

Is Gemini 3.1 Pro Preview or o3 better for coding?

o3 scores higher on coding benchmarks: 46.8 versus 42.5 in the Noometry coding category.

Which has the bigger context window?

Gemini 3.1 Pro Preview does, with 1.05M tokens against 200K.

How many benchmarks do Gemini 3.1 Pro Preview and o3 share?

45 benchmarks have published results for both models. Gemini 3.1 Pro Preview has 71 scored results on Noometry and o3 has 63.

Related comparisons

Go deeper