Noometry

Model comparison

GPT-5.1 vs Grok-3 mini

GPT-5.1 is the stronger model overall, scoring 49.0 to 41.2 on the Noometry Index.

Last verified . 29 shared benchmarks.

GPT-5.1 OpenAI

49.0

Rank #53 Confirmed

Grok-3 mini xAI

41.2

Rank #141 Confirmed

Summary

  • They share 29 benchmarks with published results for both. GPT-5.1 scores higher in 8 categories and Grok-3 mini in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where GPT-5.1 leads 39.8 to 13.6.
  • The biggest single-benchmark swing is ARC-AGI-1: 72.8% for GPT-5.1 and 16.5% for Grok-3 mini.

Side by side

GPT-5.1Grok-3 mini
ProviderOpenAIxAI
Noometry Index49.041.2
Released2025-11-132025-04-09
WeightsProprietaryProprietary
Context window400K—
Max output128K—
Input $ / M tokens$1.25—
Output $ / M tokens$10—
Results tracked6335

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-5.1 leads

GPT-5.1: 46.4 (#66), Grok-3 mini: 40.8 (#131)

BenchmarkGPT-5.1Grok-3 mini
WeirdML60.8%42.6%
LMArena Coding14541379
SWE-bench Verified68%—
SWE-bench Verified (bash only)66%—
Aider Polyglot—49.3%
LMArena WebDev1395—
SciCode43.3%—
GSO13.7%—
LiveBench Coding72.5%—
ALE-Bench1,192—

Agentic & Tool Use Not comparable

GPT-5.1: 32.7 (#60), Grok-3 mini: —

BenchmarkGPT-5.1Grok-3 mini
Terminal-Bench47.6%—
DeepResearch Bench42.8%—
LMArena Search1199—
Vending-Bench 21,473—

Reasoning GPT-5.1 leads

GPT-5.1: 39.8 (#58), Grok-3 mini: 13.6 (#334)

BenchmarkGPT-5.1Grok-3 mini
ARC-AGI-217.6%0.4%
ARC-AGI-172.8%16.5%
LMArena Hard Prompts14571375
Epoch Capabilities Index149.64140.35
SimpleBench53.2%—
Kagi LLM Benchmark—61.3%
CritPt4.9%—
Chess Puzzles32%—
EnigmaEval11.2%—
LiveBench Reasoning95.8%—
Mystery Game Puzzles19%—
DTBench90.1%—
LiveBench Data Analysis72.1%—
LMCA43.9%—
ForecastBench58.1—
LiveBench78.8%—

Math GPT-5.1 leads

GPT-5.1: 52.2 (#51), Grok-3 mini: 42.1 (#85)

BenchmarkGPT-5.1Grok-3 mini
OTIS Mock AIME 2024-202588.6%77.8%
Omni-MATH46.4%31.8%
LMArena Math14471386
FrontierMath (Feb 2025 set)31%5.9%
LiveBench Math94.5%—
MATH Level 5—90.9%
FrontierMath Tier 4 (v1)12.5%—

Knowledge GPT-5.1 leads

GPT-5.1: 50.6 (#71), Grok-3 mini: 46.4 (#81)

BenchmarkGPT-5.1Grok-3 mini
GPQA Diamond87.6%76.3%
MMLU-Pro57.9%79.9%
GPQA (HELM)44.2%67.5%
LMArena Expert14701395
Humanity's Last Exam23.7%—
SimpleQA Verified48%—
Confabulations—10.8%
Vectara Hallucination Rate10.9%—

Multimodal Not comparable

GPT-5.1: 44.8 (#19), Grok-3 mini: —

BenchmarkGPT-5.1Grok-3 mini
LMArena Vision1250—
VPCT58.7%—
LMArena Document1403—

Multilingual GPT-5.1 leads

GPT-5.1: 53.8 (#56), Grok-3 mini: 48.1 (#145)

BenchmarkGPT-5.1Grok-3 mini
LMArena Non-English14311352
LMArena Chinese14951387
LMArena French14501357
LMArena German14381349
LMArena Japanese14531342
LMArena Korean14011335
LMArena Russian14351353
LMArena Spanish14331381

Instruction Following GPT-5.1 leads

GPT-5.1: 83.9 (#1), Grok-3 mini: 78.5 (#9)

BenchmarkGPT-5.1Grok-3 mini
IFEval93.5%95.1%
LMArena Instruction Following14431357
LiveBench Instruction Following93.3%—

Long Context GPT-5.1 leads

GPT-5.1: 47.6 (#14), Grok-3 mini: 41.0 (#147)

BenchmarkGPT-5.1Grok-3 mini
LMArena Longer Query14471372
Fiction.LiveBench—66.7%
CL-bench23.7%—
CL-bench Life17.3%—

Writing & Preference GPT-5.1 leads

GPT-5.1: 64.5 (#55), Grok-3 mini: 52.5 (#169)

BenchmarkGPT-5.1Grok-3 mini
LMArena Text14431370
LMArena Creative Writing14271342
WildBench86.3%65.1%
LMArena Multi-Turn14501355
Short-Story Creative Writing—73.5%
LiveBench Language80.2%—

Frequently asked questions

Is GPT-5.1 better than Grok-3 mini?

GPT-5.1 is the stronger model overall, scoring 49.0 to 41.2 on the Noometry Index.

Is GPT-5.1 or Grok-3 mini better for coding?

GPT-5.1 scores higher on coding benchmarks: 46.4 versus 40.8 in the Noometry coding category.

How many benchmarks do GPT-5.1 and Grok-3 mini share?

29 benchmarks have published results for both models. GPT-5.1 has 63 scored results on Noometry and Grok-3 mini has 35.

Related comparisons

Go deeper