Model comparison

Devstral Small 2505 vs Phi-4

Devstral Small 2505 is the stronger model overall, scoring 34.3 to 31.2 on the Noometry Index. Phi-4 costs 1.7× less per token, which makes it the better buy when Devstral Small 2505's lead doesn't matter for your workload.

Last verified October 11, 2026. 0 shared benchmarks.

Devstral Small 2505

Mistral AI

34.3

Rank #233 Reported

Phi-4

Microsoft

31.2

Rank #279 Confirmed

Summary

The widest gap is in coding, where Devstral Small 2505 leads 38.9 to 34.4.
Phi-4 is cheaper at $0.07 / $0.14 per million input/output tokens, against $0.10 / $0.30 for Devstral Small 2505.

Side by side

Devstral Small 2505 and Phi-4 specifications
	Devstral Small 2505	Phi-4
Provider	Mistral AI	Microsoft
Noometry Index	34.3	31.2
Released	2025-05-07	2024-12-11
Weights	Open	Open
Context window	128K	128K
Max output	128K	4K
Input $ / M tokens	$0.10	$0.07
Output $ / M tokens	$0.30	$0.14
Results tracked	4	37

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Devstral Small 2505 leads

Devstral Small 2505: 38.9 (#166), Phi-4: 34.4 (#239)

Coding benchmarks
Benchmark	Devstral Small 2505	Phi-4
SWE-bench Verified (bash only)	56.4%	—
SciCode	28.8%	—
BigCodeBench Instruct	—	45.5%
LiveBench Coding	—	30.7%
LMArena Coding	—	1231
BigCodeBench Complete	—	55.4%

Agentic & Tool Use Not comparable

Devstral Small 2505: —, Phi-4: 22.8 (#128)

Agentic & Tool Use benchmarks
Benchmark	Devstral Small 2505	Phi-4
Berkeley Function Calling Leaderboard	—	28.8%
BALROG	—	11.6%

Reasoning Devstral Small 2505 leads

Devstral Small 2505: 19.7 (#252), Phi-4: 17.7 (#291)

Reasoning benchmarks
Benchmark	Devstral Small 2505	Phi-4
Kagi LLM Benchmark	37.7%	—
CritPt	0%	—
Chess Puzzles	—	1%
LiveBench Reasoning	—	47.8%
LMArena Hard Prompts	—	1220
LiveBench Data Analysis	—	45.2%
Epoch Capabilities Index	—	130.42
LiveBench	—	41.6%

Math Not comparable

Devstral Small 2505: —, Phi-4: 20.8 (#285)

Math benchmarks
Benchmark	Devstral Small 2505	Phi-4
OTIS Mock AIME 2024-2025	—	13.8%
LiveBench Math	—	42%
LMArena Math	—	1246
MATH Level 5	—	64.9%

Knowledge Not comparable

Devstral Small 2505: —, Phi-4: 32.6 (#209)

Knowledge benchmarks
Benchmark	Devstral Small 2505	Phi-4
GPQA Diamond	—	56.1%
Confabulations	—	29.4%
Vectara Hallucination Rate	—	3.7%
LMArena Expert	—	1203
MMLU	—	84.8%

Multilingual Not comparable

Devstral Small 2505: —, Phi-4: 37.2 (#237)

Multilingual benchmarks
Benchmark	Devstral Small 2505	Phi-4
LMArena Non-English	—	1197
LMArena Chinese	—	1212
LMArena French	—	1224
LMArena German	—	1222
LMArena Japanese	—	1158
LMArena Korean	—	1151
LMArena Russian	—	1209
LMArena Spanish	—	1234

Instruction Following Not comparable

Devstral Small 2505: —, Phi-4: 60.4 (#251)

Instruction Following benchmarks
Benchmark	Devstral Small 2505	Phi-4
LiveBench Instruction Following	—	58.4%
LMArena Instruction Following	—	1201

Long Context Not comparable

Devstral Small 2505: —, Phi-4: 36.9 (#226)

Long Context benchmarks
Benchmark	Devstral Small 2505	Phi-4
LMArena Longer Query	—	1217

Writing & Preference Not comparable

Devstral Small 2505: —, Phi-4: 40.5 (#244)

Writing & Preference benchmarks
Benchmark	Devstral Small 2505	Phi-4
LMArena Text	—	1217
LMArena Creative Writing	—	1182
Short-Story Creative Writing	—	62.6%
LMArena Multi-Turn	—	1206
LiveBench Language	—	25.6%

Frequently asked questions

Is Devstral Small 2505 better than Phi-4?

Which is cheaper, Devstral Small 2505 or Phi-4?

Phi-4 is cheaper. It lists at $0.07 per million input tokens and $0.14 per million output tokens; Devstral Small 2505 lists at $0.10 and $0.30.

Is Devstral Small 2505 or Phi-4 better for coding?

Devstral Small 2505 scores higher on coding benchmarks: 38.9 versus 34.4 in the Noometry coding category.

Which has the bigger context window?

Both accept 128K tokens.

How many benchmarks do Devstral Small 2505 and Phi-4 share?

0 benchmarks have published results for both models. Devstral Small 2505 has 4 scored results on Noometry and Phi-4 has 37.