Noometry

Model comparison

Gemini 1.5 Flash 8B vs Phi 3 Mini 128k Instruct

Gemini 1.5 Flash 8B and Phi 3 Mini 128k Instruct score almost the same on the Noometry Index (29.9 vs 29.7), so choose on price, context window or the category you care about most.

Last verified . 17 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.9

Rank #301 Confirmed

Phi 3 Mini 128k Instruct Microsoft

29.7

Rank #305 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Gemini 1.5 Flash 8B scores higher in 6 categories and Phi 3 Mini 128k Instruct in 2 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Phi 3 Mini 128k Instruct leads 31.6 to 14.2.
  • Phi 3 Mini 128k Instruct has downloadable open weights; the other is API-only.

Side by side

Gemini 1.5 Flash 8BPhi 3 Mini 128k Instruct
ProviderGoogleMicrosoft
Noometry Index29.929.7
Released2024-10-032024-04-23
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked2119

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 35.5 (#225), Phi 3 Mini 128k Instruct: 28.8 (#312)

BenchmarkGemini 1.5 Flash 8BPhi 3 Mini 128k Instruct
LMArena Coding12181039
BigCodeBench Instruct—29.6%
BigCodeBench Complete—40.6%

Reasoning Too close to call

Gemini 1.5 Flash 8B: 20.0 (#244), Phi 3 Mini 128k Instruct: 19.6 (#256)

BenchmarkGemini 1.5 Flash 8BPhi 3 Mini 128k Instruct
LMArena Hard Prompts12091028
DTBench50%—

Math Phi 3 Mini 128k Instruct leads

Gemini 1.5 Flash 8B: 14.2 (#302), Phi 3 Mini 128k Instruct: 31.6 (#222)

BenchmarkGemini 1.5 Flash 8BPhi 3 Mini 128k Instruct
LMArena Math12071089
OTIS Mock AIME 2024-20254.6%—

Knowledge Phi 3 Mini 128k Instruct leads

Gemini 1.5 Flash 8B: 16.0 (#289), Phi 3 Mini 128k Instruct: 26.8 (#254)

BenchmarkGemini 1.5 Flash 8BPhi 3 Mini 128k Instruct
LMArena Expert1185984
GPQA Diamond33%—

Multimodal Not comparable

Gemini 1.5 Flash 8B: 28.2 (#115), Phi 3 Mini 128k Instruct: —

BenchmarkGemini 1.5 Flash 8BPhi 3 Mini 128k Instruct
LMArena Vision1044—

Multilingual Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 38.5 (#229), Phi 3 Mini 128k Instruct: 25.2 (#285)

BenchmarkGemini 1.5 Flash 8BPhi 3 Mini 128k Instruct
LMArena Non-English12151000
LMArena Chinese12311016
LMArena French12341039
LMArena German12061006
LMArena Japanese1150899
LMArena Korean1140856
LMArena Russian12361004
LMArena Spanish12121059

Instruction Following Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 62.8 (#236), Phi 3 Mini 128k Instruct: 51.8 (#294)

BenchmarkGemini 1.5 Flash 8BPhi 3 Mini 128k Instruct
LMArena Instruction Following11991023

Long Context Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 37.0 (#225), Phi 3 Mini 128k Instruct: 30.4 (#289)

BenchmarkGemini 1.5 Flash 8BPhi 3 Mini 128k Instruct
LMArena Longer Query1219996

Writing & Preference Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 42.8 (#232), Phi 3 Mini 128k Instruct: 27.1 (#301)

BenchmarkGemini 1.5 Flash 8BPhi 3 Mini 128k Instruct
LMArena Text12261050
LMArena Creative Writing12181024
LMArena Multi-Turn1185989

Frequently asked questions

Is Gemini 1.5 Flash 8B better than Phi 3 Mini 128k Instruct?

Gemini 1.5 Flash 8B and Phi 3 Mini 128k Instruct score almost the same on the Noometry Index (29.9 vs 29.7), so choose on price, context window or the category you care about most.

Is Gemini 1.5 Flash 8B or Phi 3 Mini 128k Instruct better for coding?

Gemini 1.5 Flash 8B scores higher on coding benchmarks: 35.5 versus 28.8 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Flash 8B and Phi 3 Mini 128k Instruct share?

17 benchmarks have published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and Phi 3 Mini 128k Instruct has 19.

Related comparisons

Go deeper