Noometry

Model comparison

Gemini 1.5 Flash 8B vs Grok 4 Fast

Grok 4 Fast is the stronger model overall, scoring 39.4 to 29.9 on the Noometry Index.

Last verified . 18 shared benchmarks.

Gemini 1.5 Flash 8B Google

29.9

Rank #301 Confirmed

Grok 4 Fast xAI

39.4

Rank #167 Confirmed

Summary

  • They share 18 benchmarks with published results for both. Gemini 1.5 Flash 8B scores higher in 1 category and Grok 4 Fast in 7 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in long context, where Grok 4 Fast leads 63.2 to 37.0.
  • The biggest single-benchmark swing is DTBench: 50% for Gemini 1.5 Flash 8B and 82.7% for Grok 4 Fast.

Side by side

Gemini 1.5 Flash 8BGrok 4 Fast
ProviderGooglexAI
Noometry Index29.939.4
Released2024-10-032025-09-19
WeightsProprietaryProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked2130

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemini 1.5 Flash 8B leads

Gemini 1.5 Flash 8B: 35.5 (#225), Grok 4 Fast: 32.5 (#271)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Coding12181429
LMArena WebDev—1159
WeirdML—42.9%

Agentic & Tool Use Not comparable

Gemini 1.5 Flash 8B: —, Grok 4 Fast: 29.5 (#86)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
τ²-bench Banking—15.7%
Cybench—30%
LMArena Search—1171

Reasoning Grok 4 Fast leads

Gemini 1.5 Flash 8B: 20.0 (#244), Grok 4 Fast: 22.2 (#201)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Hard Prompts12091412
DTBench50%82.7%
ARC-AGI-2—5.3%
Kagi LLM Benchmark—66.1%
ARC-AGI-1—48.5%
Epoch Capabilities Index—144.2
ForecastBench—60.5

Math Grok 4 Fast leads

Gemini 1.5 Flash 8B: 14.2 (#302), Grok 4 Fast: 38.9 (#123)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Math12071419
OTIS Mock AIME 2024-20254.6%—

Knowledge Grok 4 Fast leads

Gemini 1.5 Flash 8B: 16.0 (#289), Grok 4 Fast: 32.0 (#214)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Expert11851411
GPQA Diamond33%—
Vectara Hallucination Rate—19.7%

Multimodal Not comparable

Gemini 1.5 Flash 8B: 28.2 (#115), Grok 4 Fast: —

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Vision1044—

Multilingual Grok 4 Fast leads

Gemini 1.5 Flash 8B: 38.5 (#229), Grok 4 Fast: 51.3 (#111)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Non-English12151396
LMArena Chinese12311457
LMArena French12341431
LMArena German12061383
LMArena Japanese11501352
LMArena Korean11401357
LMArena Russian12361389
LMArena Spanish12121415

Instruction Following Grok 4 Fast leads

Gemini 1.5 Flash 8B: 62.8 (#236), Grok 4 Fast: 73.2 (#121)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Instruction Following11991387

Long Context Grok 4 Fast leads

Gemini 1.5 Flash 8B: 37.0 (#225), Grok 4 Fast: 63.2 (#3)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Longer Query12191415
Fiction.LiveBench—94.4%

Writing & Preference Grok 4 Fast leads

Gemini 1.5 Flash 8B: 42.8 (#232), Grok 4 Fast: 60.0 (#102)

BenchmarkGemini 1.5 Flash 8BGrok 4 Fast
LMArena Text12261407
LMArena Creative Writing12181387
LMArena Multi-Turn11851414

Frequently asked questions

Is Gemini 1.5 Flash 8B better than Grok 4 Fast?

Grok 4 Fast is the stronger model overall, scoring 39.4 to 29.9 on the Noometry Index.

Is Gemini 1.5 Flash 8B or Grok 4 Fast better for coding?

Gemini 1.5 Flash 8B scores higher on coding benchmarks: 35.5 versus 32.5 in the Noometry coding category.

How many benchmarks do Gemini 1.5 Flash 8B and Grok 4 Fast share?

18 benchmarks have published results for both models. Gemini 1.5 Flash 8B has 21 scored results on Noometry and Grok 4 Fast has 30.

Related comparisons

Go deeper