Noometry

Model comparison

Gemini 1.5 Flash 8B vs o1-mini

o1-mini is the stronger model overall, scoring 33.7 to 29.7 on the Noometry Index.

Last verified . 19 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.7

Rank #292 Confirmed

o1-mini OpenAI

33.7

Rank #230 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Gemini 1.5 Flash 8B scores higher in 1 category and o1-mini in 7 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where o1-mini leads 33.7 to 13.3.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 4.6% for Gemini 1.5 Flash 8B and 46.9% for o1-mini.

Side by side

Gemini 1.5 Flash 8Bo1-mini
ProviderGoogleOpenAI
Noometry Index29.733.7
Released2024-10-032024-09-12
WeightsProprietaryProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked2139

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

Gemini 1.5 Flash 8B: 34.9 (#221), o1-mini: 35.1 (#218)

BenchmarkGemini 1.5 Flash 8Bo1-mini
LMArena Coding12181362
Aider Polyglot—32.9%
WeirdML—36.3%
LiveBench Coding—48%
HumanEval+—89%
MBPP+—78.8%

Agentic & Tool Use Not comparable

Gemini 1.5 Flash 8B: —, o1-mini: 24.3 (#112)

BenchmarkGemini 1.5 Flash 8Bo1-mini
Cybench—10%

Reasoning Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 19.2 (#242), o1-mini: 8.9 (#340)

BenchmarkGemini 1.5 Flash 8Bo1-mini
LMArena Hard Prompts12101333
ARC-AGI-2—0.8%
SimpleBench—18.1%
ARC-AGI-1—14%
LiveBench Reasoning—72.3%
DTBench50%—
LiveBench Data Analysis—57.9%
Epoch Capabilities Index—135.82
LiveBench—57.8%

Math o1-mini leads

Gemini 1.5 Flash 8B: 13.3 (#296), o1-mini: 33.7 (#168)

BenchmarkGemini 1.5 Flash 8Bo1-mini
OTIS Mock AIME 2024-20254.6%46.9%
LMArena Math12071358
LiveBench Math—62%
MATH Level 5—89.2%
FrontierMath (Feb 2025 set)—1.7%

Knowledge o1-mini leads

Gemini 1.5 Flash 8B: 16.0 (#285), o1-mini: 34.8 (#180)

BenchmarkGemini 1.5 Flash 8Bo1-mini
GPQA Diamond33%62.4%
LMArena Expert11851316
Confabulations—18.6%

Multimodal Not comparable

Gemini 1.5 Flash 8B: 27.6 (#110), o1-mini: —

BenchmarkGemini 1.5 Flash 8Bo1-mini
LMArena Vision1044—

Multilingual o1-mini leads

Gemini 1.5 Flash 8B: 38.6 (#224), o1-mini: 43.8 (#177)

BenchmarkGemini 1.5 Flash 8Bo1-mini
LMArena Non-English12151289
LMArena Chinese12311314
LMArena French12331292
LMArena German12061278
LMArena Japanese11501245
LMArena Korean11401223
LMArena Russian12361283
LMArena Spanish12121302

Instruction Following o1-mini leads

Gemini 1.5 Flash 8B: 63.1 (#231), o1-mini: 67.0 (#200)

BenchmarkGemini 1.5 Flash 8Bo1-mini
LMArena Instruction Following11991304
LiveBench Instruction Following—65.4%

Long Context o1-mini leads

Gemini 1.5 Flash 8B: 36.9 (#220), o1-mini: 40.1 (#156)

BenchmarkGemini 1.5 Flash 8Bo1-mini
LMArena Longer Query12191321

Writing & Preference o1-mini leads

Gemini 1.5 Flash 8B: 43.0 (#227), o1-mini: 48.6 (#196)

BenchmarkGemini 1.5 Flash 8Bo1-mini
LMArena Text12261317
LMArena Creative Writing12181244
LMArena Multi-Turn11851314
Short-Story Creative Writing—64.9%
LiveBench Language—40.9%

Frequently asked questions

Is Gemini 1.5 Flash 8B better than o1-mini?

o1-mini is the stronger model overall, scoring 33.7 to 29.7 on the Noometry Index.

Is Gemini 1.5 Flash 8B or o1-mini better for coding?

They score almost the same on coding (34.9 vs 35.1); test both on your own repository before choosing.

How many benchmarks do Gemini 1.5 Flash 8B and o1-mini share?

19 benchmarks have published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and o1-mini has 39.

Related comparisons

Go deeper