Noometry

Model comparison

Grok 3 vs Qwen2.5 72B Instruct

Grok 3 is the stronger model overall, scoring 39.6 to 31.5 on the Noometry Index.

Last verified . 29 shared benchmarks.

Grok 3 xAI

39.6

Rank #150 Confirmed

Qwen2.5 72B Instruct Alibaba (Qwen)

31.5

Rank #259 Confirmed

Summary

  • They share 29 benchmarks with published results for both. Grok 3 scores higher in 7 categories and Qwen2.5 72B Instruct in 2 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Grok 3 leads 46.0 to 26.9.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 55.6% for Grok 3 and 8.1% for Qwen2.5 72B Instruct.
  • Qwen2.5 72B Instruct has downloadable open weights; the other is API-only.

Side by side

Grok 3Qwen2.5 72B Instruct
ProviderxAIAlibaba (Qwen)
Noometry Index39.631.5
Released2025-04-092024-09
WeightsProprietaryOpen
Context window—131K
Max output—8K
Input $ / M tokens—$1.40
Output $ / M tokens—$5.60
Results tracked4043

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok 3 leads

Grok 3: 41.5 (#113), Qwen2.5 72B Instruct: 32.8 (#256)

BenchmarkGrok 3Qwen2.5 72B Instruct
WeirdML37.2%16%
LMArena Coding14321293
Aider Polyglot53.3%—
BigCodeBench Instruct—45.8%
BigCodeBench Complete—55.9%

Agentic & Tool Use Grok 3 leads

Grok 3: 30.2 (#69), Qwen2.5 72B Instruct: 22.0 (#124)

BenchmarkGrok 3Qwen2.5 72B Instruct
BALROG29.5%16.2%
TheAgentCompany—5.7%
METR Time Horizons—35.8%

Reasoning Qwen2.5 72B Instruct leads

Grok 3: 13.8 (#322), Qwen2.5 72B Instruct: 21.4 (#197)

BenchmarkGrok 3Qwen2.5 72B Instruct
LMArena Hard Prompts14331271
Epoch Capabilities Index138.34129.01
ARC-AGI-20%—
SimpleBench36.1%—
Kagi LLM Benchmark61.3%—
ARC-AGI-15.5%—
DTBench—62.9%
LMCA—13.4%
BIG-Bench Hard—79.8%
ForecastBench—57.5
HellaSwag—84.8%
PIQA—82.6%
WinoGrande—82.3%

Math Grok 3 leads

Grok 3: 36.2 (#122), Qwen2.5 72B Instruct: 18.1 (#281)

BenchmarkGrok 3Qwen2.5 72B Instruct
OTIS Mock AIME 2024-202555.6%8.1%
Omni-MATH46.4%33%
LMArena Math13901283
MATH Level 588.7%63.2%
FrontierMath (Feb 2025 set)3.8%—
FrontierMath Tier 4 (v1)0%—

Knowledge Grok 3 leads

Grok 3: 46.0 (#76), Qwen2.5 72B Instruct: 26.9 (#247)

BenchmarkGrok 3Qwen2.5 72B Instruct
GPQA Diamond75.8%49.1%
MMLU-Pro78.8%63.1%
Confabulations14.2%19.1%
GPQA (HELM)65%42.6%
LMArena Expert14201245
Vectara Hallucination Rate5.8%—
ARC (AI2) Challenge—94.5%
MMLU—85.3%
TriviaQA—71.9%

Multilingual Grok 3 leads

Grok 3: 52.5 (#81), Qwen2.5 72B Instruct: 41.2 (#208)

BenchmarkGrok 3Qwen2.5 72B Instruct
LMArena Non-English14111252
LMArena Chinese14481272
LMArena French14601279
LMArena German14311234
LMArena Japanese13891180
LMArena Korean13741188
LMArena Russian14161264
LMArena Spanish14171255

Instruction Following Grok 3 leads

Grok 3: 75.2 (#69), Qwen2.5 72B Instruct: 65.7 (#216)

BenchmarkGrok 3Qwen2.5 72B Instruct
IFEval88.4%80.6%
LMArena Instruction Following14091254

Long Context Too close to call

Grok 3: 38.7 (#187), Qwen2.5 72B Instruct: 38.9 (#183)

BenchmarkGrok 3Qwen2.5 72B Instruct
LMArena Longer Query14391282
Fiction.LiveBench58.3%—

Writing & Preference Grok 3 leads

Grok 3: 56.0 (#136), Qwen2.5 72B Instruct: 46.9 (#210)

BenchmarkGrok 3Qwen2.5 72B Instruct
LMArena Text14261269
LMArena Creative Writing14141221
WildBench84.9%80.2%
LMArena Multi-Turn14241272
Short-Story Creative Writing76.4%—
EQ-Bench Creative Writing1186—

Frequently asked questions

Is Grok 3 better than Qwen2.5 72B Instruct?

Grok 3 is the stronger model overall, scoring 39.6 to 31.5 on the Noometry Index.

Is Grok 3 or Qwen2.5 72B Instruct better for coding?

Grok 3 scores higher on coding benchmarks: 41.5 versus 32.8 in the Noometry coding category.

How many benchmarks do Grok 3 and Qwen2.5 72B Instruct share?

29 benchmarks have published results for both models. Grok 3 has 40 scored results on Noometry and Qwen2.5 72B Instruct has 43.

Related comparisons

Go deeper