Noometry

Model comparison

Grok-2 (Dec 2024) vs Phi-4

Grok-2 (Dec 2024) is the stronger model overall, scoring 33.7 to 31.2 on the Noometry Index.

Last verified . 30 shared benchmarks.

Grok-2 (Dec 2024) xAI

33.7

Rank #239 Confirmed

Phi-4 Microsoft

31.2

Rank #279 Confirmed

Summary

  • They share 30 benchmarks with published results for both. Grok-2 (Dec 2024) scores higher in 5 categories and Phi-4 in 3 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Grok-2 (Dec 2024) leads 48.6 to 40.5.
  • The biggest single-benchmark swing is LiveBench Language: 45.6% for Grok-2 (Dec 2024) and 25.6% for Phi-4.
  • Phi-4 has downloadable open weights; the other is API-only.

Side by side

Grok-2 (Dec 2024)Phi-4
ProviderxAIMicrosoft
Noometry Index33.731.2
Released2024-08-132024-12-11
WeightsProprietaryOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.07
Output $ / M tokens—$0.14
Results tracked3437

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Phi-4 leads

Grok-2 (Dec 2024): 33.3 (#258), Phi-4: 34.4 (#239)

BenchmarkGrok-2 (Dec 2024)Phi-4
LiveBench Coding46.4%30.7%
LMArena Coding12871231
WeirdML22.2%—
BigCodeBench Instruct—45.5%
BigCodeBench Complete—55.4%

Agentic & Tool Use Not comparable

Grok-2 (Dec 2024): —, Phi-4: 22.8 (#128)

BenchmarkGrok-2 (Dec 2024)Phi-4
Berkeley Function Calling Leaderboard—28.8%
BALROG—11.6%

Reasoning Too close to call

Grok-2 (Dec 2024): 16.9 (#299), Phi-4: 17.7 (#291)

BenchmarkGrok-2 (Dec 2024)Phi-4
LiveBench Reasoning54.8%47.8%
LMArena Hard Prompts12721220
LiveBench Data Analysis54.5%45.2%
Epoch Capabilities Index130.48130.42
LiveBench54.3%41.6%
SimpleBench22.7%—
Chess Puzzles—1%
DTBench65.2%—

Math Too close to call

Grok-2 (Dec 2024): 20.8 (#284), Phi-4: 20.8 (#285)

BenchmarkGrok-2 (Dec 2024)Phi-4
OTIS Mock AIME 2024-202511.5%13.8%
LiveBench Math54.9%42%
LMArena Math12831246
MATH Level 563.5%64.9%
FrontierMath (Feb 2025 set)0.7%—

Knowledge Phi-4 leads

Grok-2 (Dec 2024): 29.8 (#233), Phi-4: 32.6 (#209)

BenchmarkGrok-2 (Dec 2024)Phi-4
GPQA Diamond53.8%56.1%
Confabulations20.1%29.4%
LMArena Expert12541203
Vectara Hallucination Rate—3.7%
MMLU—84.8%

Multilingual Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 43.1 (#188), Phi-4: 37.2 (#237)

BenchmarkGrok-2 (Dec 2024)Phi-4
LMArena Non-English12821197
LMArena Chinese12891212
LMArena French13181224
LMArena German12871222
LMArena Japanese12441158
LMArena Korean12371151
LMArena Russian12861209
LMArena Spanish12811234

Instruction Following Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 66.9 (#202), Phi-4: 60.4 (#251)

BenchmarkGrok-2 (Dec 2024)Phi-4
LiveBench Instruction Following69.6%58.4%
LMArena Instruction Following12701201

Long Context Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 38.8 (#190), Phi-4: 36.9 (#226)

BenchmarkGrok-2 (Dec 2024)Phi-4
LMArena Longer Query12761217

Writing & Preference Grok-2 (Dec 2024) leads

Grok-2 (Dec 2024): 48.6 (#198), Phi-4: 40.5 (#244)

BenchmarkGrok-2 (Dec 2024)Phi-4
LMArena Text13051217
LMArena Creative Writing12841182
Short-Story Creative Writing63.6%62.6%
LMArena Multi-Turn12901206
LiveBench Language45.6%25.6%

Frequently asked questions

Is Grok-2 (Dec 2024) better than Phi-4?

Grok-2 (Dec 2024) is the stronger model overall, scoring 33.7 to 31.2 on the Noometry Index.

Is Grok-2 (Dec 2024) or Phi-4 better for coding?

Phi-4 scores higher on coding benchmarks: 34.4 versus 33.3 in the Noometry coding category.

How many benchmarks do Grok-2 (Dec 2024) and Phi-4 share?

30 benchmarks have published results for both models. Grok-2 (Dec 2024) has 34 scored results on Noometry and Phi-4 has 37.

Related comparisons

Go deeper