Noometry

Model comparison

Mistral Small 3 vs Phi-4

Mistral Small 3 and Phi-4 score almost the same on the Noometry Index (31.2 vs 31.2), so choose on price, context window or the category you care about most.

Last verified . 23 shared benchmarks.

Mistral Small 3 Mistral AI

31.2

Rank #278 Confirmed

Phi-4 Microsoft

31.2

Rank #279 Confirmed

Summary

  • They share 23 benchmarks with published results for both. Mistral Small 3 scores higher in 5 categories and Phi-4 in 3 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Phi-4 leads 40.5 to 32.2.
  • The biggest single-benchmark swing is GPQA Diamond: 47.3% for Mistral Small 3 and 56.1% for Phi-4.
  • Mistral Small 3 is cheaper at $0.05 / $0.08 per million input/output tokens, against $0.07 / $0.14 for Phi-4.
  • Phi-4 accepts more context: 128K tokens versus 33K.

Side by side

Mistral Small 3Phi-4
ProviderMistral AIMicrosoft
Noometry Index31.231.2
Released2025-01-302024-12-11
WeightsOpenOpen
Context window33K128K
Max output16K4K
Input $ / M tokens$0.05$0.07
Output $ / M tokens$0.08$0.14
Results tracked2437

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Small 3 leads

Mistral Small 3: 36.5 (#207), Phi-4: 34.4 (#239)

BenchmarkMistral Small 3Phi-4
BigCodeBench Instruct45.3%45.5%
LMArena Coding12461231
BigCodeBench Complete50.4%55.4%
LiveBench Coding—30.7%

Agentic & Tool Use Not comparable

Mistral Small 3: —, Phi-4: 22.8 (#128)

BenchmarkMistral Small 3Phi-4
Berkeley Function Calling Leaderboard—28.8%
BALROG—11.6%

Reasoning Mistral Small 3 leads

Mistral Small 3: 18.9 (#273), Phi-4: 17.7 (#291)

BenchmarkMistral Small 3Phi-4
Chess Puzzles0%1%
LMArena Hard Prompts12331220
Epoch Capabilities Index127.07130.42
LiveBench Reasoning—47.8%
LiveBench Data Analysis—45.2%
LiveBench—41.6%

Math Phi-4 leads

Mistral Small 3: 16.3 (#295), Phi-4: 20.8 (#285)

BenchmarkMistral Small 3Phi-4
OTIS Mock AIME 2024-20256.7%13.8%
LMArena Math12401246
LiveBench Math—42%
MATH Level 5—64.9%

Knowledge Phi-4 leads

Mistral Small 3: 25.1 (#263), Phi-4: 32.6 (#209)

BenchmarkMistral Small 3Phi-4
GPQA Diamond47.3%56.1%
Confabulations25.2%29.4%
LMArena Expert12021203
Vectara Hallucination Rate—3.7%
MMLU—84.8%

Multilingual Too close to call

Mistral Small 3: 37.3 (#236), Phi-4: 37.2 (#237)

BenchmarkMistral Small 3Phi-4
LMArena Non-English11981197
LMArena Chinese12041212
LMArena French12031224
LMArena German12111222
LMArena Japanese11111158
LMArena Korean11881151
LMArena Russian12161209
LMArena Spanish—1234

Instruction Following Mistral Small 3 leads

Mistral Small 3: 63.7 (#229), Phi-4: 60.4 (#251)

BenchmarkMistral Small 3Phi-4
LMArena Instruction Following12141201
LiveBench Instruction Following—58.4%

Long Context Too close to call

Mistral Small 3: 37.8 (#211), Phi-4: 36.9 (#226)

BenchmarkMistral Small 3Phi-4
LMArena Longer Query12461217

Writing & Preference Phi-4 leads

Mistral Small 3: 32.2 (#280), Phi-4: 40.5 (#244)

BenchmarkMistral Small 3Phi-4
LMArena Text12341217
LMArena Creative Writing11951182
LMArena Multi-Turn12171206
Short-Story Creative Writing—62.6%
EQ-Bench Creative Writing707—
LiveBench Language—25.6%

Frequently asked questions

Is Mistral Small 3 better than Phi-4?

Mistral Small 3 and Phi-4 score almost the same on the Noometry Index (31.2 vs 31.2), so choose on price, context window or the category you care about most.

Which is cheaper, Mistral Small 3 or Phi-4?

Mistral Small 3 is cheaper. It lists at $0.05 per million input tokens and $0.08 per million output tokens; Phi-4 lists at $0.07 and $0.14.

Is Mistral Small 3 or Phi-4 better for coding?

Mistral Small 3 scores higher on coding benchmarks: 36.5 versus 34.4 in the Noometry coding category.

Which has the bigger context window?

Phi-4 does, with 128K tokens against 33K.

How many benchmarks do Mistral Small 3 and Phi-4 share?

23 benchmarks have published results for both models. Mistral Small 3 has 24 scored results on Noometry and Phi-4 has 37.

Related comparisons

Go deeper