Noometry

Model comparison

Grok 4 vs o1-mini

Grok 4 is the stronger model overall, scoring 48.1 to 34.0 on the Noometry Index.

Last verified . 29 shared benchmarks.

Grok 4 xAI

48.1

Rank #56 Confirmed

o1-mini OpenAI

34.0

Rank #235 Confirmed

Summary

  • They share 29 benchmarks with published results for both. Grok 4 scores higher in 9 categories and o1-mini in 0 categories; 9 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Grok 4 leads 36.7 to 8.8.
  • The biggest single-benchmark swing is ARC-AGI-1: 66.7% for Grok 4 and 14% for o1-mini.

Side by side

Grok 4o1-mini
ProviderxAIOpenAI
Noometry Index48.134.0
Released2025-07-092024-09-12
WeightsProprietaryProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked4839

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok 4 leads

Grok 4: 50.3 (#46), o1-mini: 35.5 (#224)

BenchmarkGrok 4o1-mini
Aider Polyglot79.6%32.9%
WeirdML45.7%36.3%
LMArena Coding14081362
LiveBench Coding—48%
HumanEval+—89%
MBPP+—78.8%

Agentic & Tool Use Grok 4 leads

Grok 4: 32.3 (#68), o1-mini: 24.6 (#118)

BenchmarkGrok 4o1-mini
Cybench43%10%
Terminal-Bench27.2%—
Berkeley Function Calling Leaderboard63%—
GDPval21.1%—
DeepResearch Bench47.3%—
BALROG43.6%—
LMArena Search1142—
METR Time Horizons66.6%—

Reasoning Grok 4 leads

Grok 4: 36.7 (#65), o1-mini: 8.8 (#346)

BenchmarkGrok 4o1-mini
ARC-AGI-216%0.8%
SimpleBench60.5%18.1%
ARC-AGI-166.7%14%
LMArena Hard Prompts14091333
Epoch Capabilities Index146.44135.82
Kagi LLM Benchmark73.6%—
Chess Puzzles28%—
LiveBench Reasoning—72.3%
LiveBench Data Analysis—57.9%
ForecastBench60.9—
LiveBench—57.8%

Math Grok 4 leads

Grok 4: 48.4 (#64), o1-mini: 35.4 (#186)

BenchmarkGrok 4o1-mini
OTIS Mock AIME 2024-202584%46.9%
LMArena Math14221358
FrontierMath (Feb 2025 set)19.7%1.7%
Omni-MATH60.3%—
LiveBench Math—62%
MATH Level 5—89.2%
FrontierMath Tier 4 (v1)2.1%—

Knowledge Grok 4 leads

Grok 4: 53.8 (#55), o1-mini: 34.9 (#192)

BenchmarkGrok 4o1-mini
GPQA Diamond87%62.4%
Confabulations12.4%18.6%
LMArena Expert14151316
MMLU-Pro85.1%—
GPQA (HELM)72.7%—

Multimodal Not comparable

Grok 4: 33.7 (#94), o1-mini: —

BenchmarkGrok 4o1-mini
LMArena Vision1210—
GeoBench45%—

Multilingual Grok 4 leads

Grok 4: 51.8 (#103), o1-mini: 43.6 (#182)

BenchmarkGrok 4o1-mini
LMArena Non-English14031289
LMArena Chinese14271314
LMArena French14181293
LMArena German14291278
LMArena Japanese13941245
LMArena Korean13771223
LMArena Russian14101283
LMArena Spanish14201303

Instruction Following Grok 4 leads

Grok 4: 79.2 (#5), o1-mini: 66.7 (#206)

BenchmarkGrok 4o1-mini
LMArena Instruction Following13871304
LiveBench Instruction Following—65.4%
IFEval94.9%—

Long Context Grok 4 leads

Grok 4: 63.1 (#4), o1-mini: 40.1 (#161)

BenchmarkGrok 4o1-mini
LMArena Longer Query14091320
Fiction.LiveBench94.4%—

Writing & Preference Grok 4 leads

Grok 4: 58.5 (#116), o1-mini: 48.4 (#202)

BenchmarkGrok 4o1-mini
LMArena Text14111317
LMArena Creative Writing13971244
Short-Story Creative Writing76.9%64.9%
LMArena Multi-Turn14161314
WildBench79.7%—
LiveBench Language—40.9%

Frequently asked questions

Is Grok 4 better than o1-mini?

Grok 4 is the stronger model overall, scoring 48.1 to 34.0 on the Noometry Index.

Is Grok 4 or o1-mini better for coding?

Grok 4 scores higher on coding benchmarks: 50.3 versus 35.5 in the Noometry coding category.

How many benchmarks do Grok 4 and o1-mini share?

29 benchmarks have published results for both models. Grok 4 has 48 scored results on Noometry and o1-mini has 39.

Related comparisons

Go deeper