Noometry

Model comparison

Gemini 1.5 Flash 8B vs Phi-4

Phi-4 is the stronger model overall, scoring 30.9 to 29.7 on the Noometry Index.

Last verified . 19 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.7

Rank #292 Confirmed

Phi-4 Microsoft

30.9

Rank #269 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Gemini 1.5 Flash 8B scores higher in 6 categories and Phi-4 in 2 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Phi-4 leads 32.5 to 16.0.
  • The biggest single-benchmark swing is GPQA Diamond: 33% for Gemini 1.5 Flash 8B and 56.1% for Phi-4.
  • Phi-4 has downloadable open weights; the other is API-only.

Side by side

Gemini 1.5 Flash 8BPhi-4
ProviderGoogleMicrosoft
Noometry Index29.730.9
Released2024-10-032024-12-11
WeightsProprietaryOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.07
Output $ / M tokens—$0.14
Results tracked2137

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 34.9 (#221), Phi-4: 33.8 (#237)

BenchmarkGemini 1.5 Flash 8BPhi-4
LMArena Coding12181231
BigCodeBench Instruct—45.5%
LiveBench Coding—30.7%
BigCodeBench Complete—55.4%

Agentic & Tool Use Not comparable

Gemini 1.5 Flash 8B: —, Phi-4: 22.6 (#120)

BenchmarkGemini 1.5 Flash 8BPhi-4
Berkeley Function Calling Leaderboard—28.8%
BALROG—11.6%

Reasoning Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 19.2 (#242), Phi-4: 17.0 (#287)

BenchmarkGemini 1.5 Flash 8BPhi-4
LMArena Hard Prompts12101220
Chess Puzzles—1%
LiveBench Reasoning—47.8%
DTBench50%—
LiveBench Data Analysis—45.2%
Epoch Capabilities Index—130.43
LiveBench—41.6%

Math Phi-4 leads

Gemini 1.5 Flash 8B: 13.3 (#296), Phi-4: 19.6 (#279)

BenchmarkGemini 1.5 Flash 8BPhi-4
OTIS Mock AIME 2024-20254.6%13.8%
LMArena Math12071246
LiveBench Math—42%
MATH Level 5—64.9%

Knowledge Phi-4 leads

Gemini 1.5 Flash 8B: 16.0 (#285), Phi-4: 32.5 (#204)

BenchmarkGemini 1.5 Flash 8BPhi-4
GPQA Diamond33%56.1%
LMArena Expert11851203
Confabulations—29.4%
Vectara Hallucination Rate—3.7%
MMLU—84.8%

Multimodal Not comparable

Gemini 1.5 Flash 8B: 27.6 (#110), Phi-4: —

BenchmarkGemini 1.5 Flash 8BPhi-4
LMArena Vision1044—

Multilingual Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 38.6 (#224), Phi-4: 37.4 (#232)

BenchmarkGemini 1.5 Flash 8BPhi-4
LMArena Non-English12151197
LMArena Chinese12311211
LMArena French12331223
LMArena German12061222
LMArena Japanese11501158
LMArena Korean11401151
LMArena Russian12361209
LMArena Spanish12121234

Instruction Following Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 63.1 (#231), Phi-4: 60.6 (#246)

BenchmarkGemini 1.5 Flash 8BPhi-4
LMArena Instruction Following11991201
LiveBench Instruction Following—58.4%

Long Context Too close to call

Gemini 1.5 Flash 8B: 36.9 (#220), Phi-4: 36.9 (#221)

BenchmarkGemini 1.5 Flash 8BPhi-4
LMArena Longer Query12191217

Writing & Preference Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 43.0 (#227), Phi-4: 40.7 (#239)

BenchmarkGemini 1.5 Flash 8BPhi-4
LMArena Text12261217
LMArena Creative Writing12181182
LMArena Multi-Turn11851206
Short-Story Creative Writing—62.6%
LiveBench Language—25.6%

Frequently asked questions

Is Gemini 1.5 Flash 8B better than Phi-4?

Phi-4 is the stronger model overall, scoring 30.9 to 29.7 on the Noometry Index.

Is Gemini 1.5 Flash 8B or Phi-4 better for coding?

Gemini 1.5 Flash 8B scores higher on coding benchmarks: 34.9 versus 33.8 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Flash 8B and Phi-4 share?

19 benchmarks have published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and Phi-4 has 37.

Related comparisons

Go deeper