Noometry

Model comparison

Grok-2 (Dec 2024) vs Ministral 8B

Grok-2 (Dec 2024) is the stronger model overall, scoring 33.4 to 27.8 on the Noometry Index.

Last verified . 15 shared benchmarks.

Grok-2 (Dec 2024) xAI

33.4

Rank #233 Confirmed

Ministral 8B Mistral AI

27.8

Rank #321 Confirmed

Summary

  • They share 15 benchmarks with published results for both. Grok-2 (Dec 2024) scores higher in 5 categories and Ministral 8B in 3 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Grok-2 (Dec 2024) leads 29.8 to 12.5.
  • The biggest single-benchmark swing is MATH Level 5: 63.5% for Grok-2 (Dec 2024) and 14.9% for Ministral 8B.
  • Ministral 8B has downloadable open weights; the other is API-only.

Side by side

Grok-2 (Dec 2024)Ministral 8B
ProviderxAIMistral AI
Noometry Index33.427.8
Released2024-08-132024-10-01
WeightsProprietaryOpen
Context window—128K
Max output—128K
Input $ / M tokens—$0.10
Output $ / M tokens—$0.10
Results tracked3417

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Ministral 8B leads

Grok-2 (Dec 2024): 32.9 (#254), Ministral 8B: 34.4 (#226)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
LMArena Coding12881202
WeirdML22.2%—
LiveBench Coding46.4%—

Agentic & Tool Use Not comparable

Grok-2 (Dec 2024): —, Ministral 8B: 16.2 (#140)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
Berkeley Function Calling Leaderboard—11.1%

Reasoning Ministral 8B leads

Grok-2 (Dec 2024): 16.4 (#294), Ministral 8B: 17.7 (#279)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
LMArena Hard Prompts12721191
DTBench65.2%45.7%
SimpleBench22.7%—
LiveBench Reasoning54.8%—
LiveBench Data Analysis54.5%—
Epoch Capabilities Index130.48—
LiveBench54.3%—

Math Ministral 8B leads

Grok-2 (Dec 2024): 19.6 (#278), Ministral 8B: 23.7 (#262)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
LMArena Math12831188
MATH Level 563.5%14.9%
OTIS Mock AIME 2024-202511.5%—
LiveBench Math54.9%—
FrontierMath (Feb 2025 set)0.7%—

Knowledge Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 29.8 (#225), Ministral 8B: 12.5 (#292)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
GPQA Diamond53.8%27.1%
LMArena Expert12541170
Confabulations20.1%—
Vectara Hallucination Rate—7.4%

Multilingual Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 43.3 (#183), Ministral 8B: 35.3 (#242)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
LMArena Non-English12821165
LMArena Chinese12891193
LMArena Russian12861195
LMArena French1317—
LMArena German1287—
LMArena Japanese1244—
LMArena Korean1237—
LMArena Spanish1281—

Instruction Following Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 67.1 (#197), Ministral 8B: 60.7 (#245)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
LMArena Instruction Following12701161
LiveBench Instruction Following69.6%—

Long Context Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 38.7 (#186), Ministral 8B: 36.7 (#222)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
LMArena Longer Query12761212

Writing & Preference Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 48.8 (#192), Ministral 8B: 39.8 (#241)

BenchmarkGrok-2 (Dec 2024)Ministral 8B
LMArena Text13051191
LMArena Creative Writing12841175
LMArena Multi-Turn12901166
Short-Story Creative Writing63.6%—
LiveBench Language45.6%—

Frequently asked questions

Is Grok-2 (Dec 2024) better than Ministral 8B?

Grok-2 (Dec 2024) is the stronger model overall, scoring 33.4 to 27.8 on the Noometry Index.

Is Grok-2 (Dec 2024) or Ministral 8B better for coding?

Ministral 8B scores higher on coding benchmarks: 34.4 versus 32.9 in the Noometry coding category.

How many benchmarks do Grok-2 (Dec 2024) and Ministral 8B share?

15 benchmarks have published results for both models. Grok-2 (Dec 2024) has 34 scored results on Noometry and Ministral 8B has 17.

Related comparisons

Go deeper