Noometry

Model comparison

GPT-5.1 vs o1-mini

GPT-5.1 is the stronger model overall, scoring 48.8 to 33.7 on the Noometry Index.

Last verified . 32 shared benchmarks.

GPT-5.1 OpenAI

48.8

Rank #47 Confirmed

o1-mini OpenAI

33.7

Rank #230 Confirmed

Summary

  • They share 32 benchmarks with published results for both. GPT-5.1 scores higher in 9 categories and o1-mini in 0 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where GPT-5.1 leads 39.9 to 8.9.
  • The biggest single-benchmark swing is ARC-AGI-1: 72.8% for GPT-5.1 and 14% for o1-mini.

Side by side

GPT-5.1o1-mini
ProviderOpenAIOpenAI
Noometry Index48.833.7
Released2025-11-132024-09-12
WeightsProprietaryProprietary
Context window400K—
Max output128K—
Input $ / M tokens$1.25—
Output $ / M tokens$10—
Results tracked6339

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.1 leads

GPT-5.1: 46.9 (#59), o1-mini: 35.1 (#218)

BenchmarkGPT-5.1o1-mini
WeirdML60.8%36.3%
LiveBench Coding72.5%48%
LMArena Coding14541362
SWE-bench Verified68%—
SWE-bench Verified (bash only)66%—
Aider Polyglot—32.9%
LMArena WebDev1395—
SciCode43.3%—
GSO13.7%—
ALE-Bench1,192—
HumanEval+—89%
MBPP+—78.8%

Agentic & Tool Use GPT-5.1 leads

GPT-5.1: 32.5 (#55), o1-mini: 24.3 (#112)

BenchmarkGPT-5.1o1-mini
Terminal-Bench47.6%—
Cybench—10%
DeepResearch Bench42.8%—
LMArena Search1199—
Vending-Bench 21,473—

Reasoning GPT-5.1 leads

GPT-5.1: 39.9 (#50), o1-mini: 8.9 (#340)

BenchmarkGPT-5.1o1-mini
ARC-AGI-217.6%0.8%
SimpleBench53.2%18.1%
ARC-AGI-172.8%14%
LiveBench Reasoning95.8%72.3%
LMArena Hard Prompts14571333
LiveBench Data Analysis72.1%57.9%
Epoch Capabilities Index149.66135.82
LiveBench78.8%57.8%
CritPt4.9%—
Chess Puzzles32%—
EnigmaEval11.2%—
Mystery Game Puzzles19%—
DTBench90.1%—
LMCA43.9%—
ForecastBench58.1—

Math GPT-5.1 leads

GPT-5.1: 50.3 (#45), o1-mini: 33.7 (#168)

BenchmarkGPT-5.1o1-mini
OTIS Mock AIME 2024-202588.6%46.9%
LiveBench Math94.5%62%
LMArena Math14461358
FrontierMath (Feb 2025 set)31%1.7%
Omni-MATH46.4%—
MATH Level 5—89.2%
FrontierMath Tier 4 (v1)12.5%—

Knowledge GPT-5.1 leads

GPT-5.1: 50.5 (#65), o1-mini: 34.8 (#180)

BenchmarkGPT-5.1o1-mini
GPQA Diamond87.6%62.4%
LMArena Expert14701316
Humanity's Last Exam23.7%—
SimpleQA Verified48%—
MMLU-Pro57.9%—
Confabulations—18.6%
Vectara Hallucination Rate10.9%—
GPQA (HELM)44.2%—

Multimodal Not comparable

GPT-5.1: 44.0 (#17), o1-mini: —

BenchmarkGPT-5.1o1-mini
LMArena Vision1250—
VPCT58.7%—
LMArena Document1403—

Multilingual GPT-5.1 leads

GPT-5.1: 54.0 (#51), o1-mini: 43.8 (#177)

BenchmarkGPT-5.1o1-mini
LMArena Non-English14311289
LMArena Chinese14951314
LMArena French14511292
LMArena German14391278
LMArena Japanese14531245
LMArena Korean14011223
LMArena Russian14351283
LMArena Spanish14301302

Instruction Following GPT-5.1 leads

GPT-5.1: 84.0 (#1), o1-mini: 67.0 (#200)

BenchmarkGPT-5.1o1-mini
LiveBench Instruction Following93.3%65.4%
LMArena Instruction Following14421304
IFEval93.5%—

Long Context GPT-5.1 leads

GPT-5.1: 47.6 (#14), o1-mini: 40.1 (#156)

BenchmarkGPT-5.1o1-mini
LMArena Longer Query14481321
CL-bench23.7%—
CL-bench Life17.3%—

Writing & Preference GPT-5.1 leads

GPT-5.1: 64.7 (#53), o1-mini: 48.6 (#196)

BenchmarkGPT-5.1o1-mini
LMArena Text14431317
LMArena Creative Writing14271244
LMArena Multi-Turn14491314
LiveBench Language80.2%40.9%
Short-Story Creative Writing—64.9%
WildBench86.3%—

Frequently asked questions

Is GPT-5.1 better than o1-mini?

GPT-5.1 is the stronger model overall, scoring 48.8 to 33.7 on the Noometry Index.

Is GPT-5.1 or o1-mini better for coding?

GPT-5.1 scores higher on coding benchmarks: 46.9 versus 35.1 in the Noometry coding category.

How many benchmarks do GPT-5.1 and o1-mini share?

32 benchmarks have published results for both models. GPT-5.1 has 63 scored results on Noometry and o1-mini has 39.

Related comparisons

Go deeper