Noometry

Model comparison

Olmo 2 0325 32b Instruct vs Phi-4

Olmo 2 0325 32b Instruct is the stronger model overall, scoring 32.7 to 31.2 on the Noometry Index.

Last verified . 11 shared benchmarks.

Olmo 2 0325 32b Instruct Allen Institute for AI (Ai2)

32.7

Rank #254 Confirmed

Phi-4 Microsoft

31.2

Rank #279 Confirmed

Summary

  • They share 11 benchmarks with published results for both. Olmo 2 0325 32b Instruct scores higher in 5 categories and Phi-4 in 3 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Phi-4 leads 32.6 to 19.5.

Side by side

Olmo 2 0325 32b InstructPhi-4
ProviderAllen Institute for AI (Ai2)Microsoft
Noometry Index32.731.2
Released—2024-12-11
WeightsOpenOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.07
Output $ / M tokens—$0.14
Results tracked1637

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

Olmo 2 0325 32b Instruct: 35.2 (#227), Phi-4: 34.4 (#239)

BenchmarkOlmo 2 0325 32b InstructPhi-4
LMArena Coding12101231
BigCodeBench Instruct—45.5%
LiveBench Coding—30.7%
BigCodeBench Complete—55.4%

Agentic & Tool Use Not comparable

Olmo 2 0325 32b Instruct: —, Phi-4: 22.8 (#128)

BenchmarkOlmo 2 0325 32b InstructPhi-4
Berkeley Function Calling Leaderboard—28.8%
BALROG—11.6%

Reasoning Olmo 2 0325 32b Instruct leads

Olmo 2 0325 32b Instruct: 23.6 (#175), Phi-4: 17.7 (#291)

BenchmarkOlmo 2 0325 32b InstructPhi-4
LMArena Hard Prompts12081220
Chess Puzzles—1%
LiveBench Reasoning—47.8%
LiveBench Data Analysis—45.2%
Epoch Capabilities Index—130.42
LiveBench—41.6%

Math Olmo 2 0325 32b Instruct leads

Olmo 2 0325 32b Instruct: 26.8 (#255), Phi-4: 20.8 (#285)

BenchmarkOlmo 2 0325 32b InstructPhi-4
LMArena Math12081246
OTIS Mock AIME 2024-2025—13.8%
Omni-MATH16.1%—
LiveBench Math—42%
MATH Level 5—64.9%

Knowledge Phi-4 leads

Olmo 2 0325 32b Instruct: 19.5 (#279), Phi-4: 32.6 (#209)

BenchmarkOlmo 2 0325 32b InstructPhi-4
GPQA Diamond—56.1%
MMLU-Pro41.4%—
Confabulations—29.4%
Vectara Hallucination Rate—3.7%
GPQA (HELM)28.7%—
LMArena Expert—1203
MMLU—84.8%

Multilingual Phi-4 leads

Olmo 2 0325 32b Instruct: 34.8 (#248), Phi-4: 37.2 (#237)

BenchmarkOlmo 2 0325 32b InstructPhi-4
LMArena Non-English11601197
LMArena Chinese11921212
LMArena Russian11871209
LMArena French—1224
LMArena German—1222
LMArena Japanese—1158
LMArena Korean—1151
LMArena Spanish—1234

Instruction Following Olmo 2 0325 32b Instruct leads

Olmo 2 0325 32b Instruct: 61.5 (#244), Phi-4: 60.4 (#251)

BenchmarkOlmo 2 0325 32b InstructPhi-4
LMArena Instruction Following11861201
LiveBench Instruction Following—58.4%
IFEval78%—

Long Context Too close to call

Olmo 2 0325 32b Instruct: 36.2 (#234), Phi-4: 36.9 (#226)

BenchmarkOlmo 2 0325 32b InstructPhi-4
LMArena Longer Query11941217

Writing & Preference Olmo 2 0325 32b Instruct leads

Olmo 2 0325 32b Instruct: 42.1 (#236), Phi-4: 40.5 (#244)

BenchmarkOlmo 2 0325 32b InstructPhi-4
LMArena Text12181217
LMArena Creative Writing11991182
LMArena Multi-Turn12211206
Short-Story Creative Writing—62.6%
WildBench73.4%—
LiveBench Language—25.6%

Frequently asked questions

Is Olmo 2 0325 32b Instruct better than Phi-4?

Olmo 2 0325 32b Instruct is the stronger model overall, scoring 32.7 to 31.2 on the Noometry Index.

Is Olmo 2 0325 32b Instruct or Phi-4 better for coding?

They score almost the same on coding (35.2 vs 34.4); test both on your own repository before choosing.

How many benchmarks do Olmo 2 0325 32b Instruct and Phi-4 share?

11 benchmarks have published results for both models. Olmo 2 0325 32b Instruct has 16 scored results on Noometry and Phi-4 has 37.

Related comparisons

Go deeper