Noometry

Model comparison

Gemma 3 1B vs Llama 2-70B

Llama 2-70B is the stronger model overall, scoring 24.4 to 21.1 on the Noometry Index.

Last verified . 2 shared benchmarks.

Gemma 3 1B Google

21.1

Rank #353 Reported

Llama 2-70B Meta

24.4

Rank #349 Confirmed

Summary

  • They share 2 benchmarks with published results for both. Gemma 3 1B scores higher in 2 categories and Llama 2-70B in 1 category; 2 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Gemma 3 1B leads 19.2 to 14.4.
  • The biggest single-benchmark swing is GPQA Diamond: 19.9% for Gemma 3 1B and 26.3% for Llama 2-70B.

Side by side

Gemma 3 1BLlama 2-70B
ProviderGoogleMeta
Noometry Index21.124.4
Released2025-03-122023-07-18
WeightsOpenOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked435

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Gemma 3 1B: —, Llama 2-70B: 31.4 (#286)

BenchmarkGemma 3 1BLlama 2-70B
LMArena Coding—1079

Agentic & Tool Use Not comparable

Gemma 3 1B: 13.7 (#152), Llama 2-70B: —

BenchmarkGemma 3 1BLlama 2-70B
Berkeley Function Calling Leaderboard7.2%—

Reasoning Gemma 3 1B leads

Gemma 3 1B: 19.2 (#264), Llama 2-70B: 14.4 (#325)

BenchmarkGemma 3 1BLlama 2-70B
Chess Puzzles0%—
LMArena Hard Prompts—1073
DTBench—41.6%
BIG-Bench Hard—64.9%
CommonsenseQA 2.0—50%
Epoch Capabilities Index—113.79
ForecastBench—51.4
HellaSwag—85.3%
LAMBADA—78.9%
PIQA—82.8%
WinoGrande—80.2%

Math Gemma 3 1B leads

Gemma 3 1B: 10.2 (#316), Llama 2-70B: 8.1 (#326)

BenchmarkGemma 3 1BLlama 2-70B
OTIS Mock AIME 2024-20251.1%0%
LMArena Math—1091
MATH Level 5—3.3%
GSM8K—69.6%

Knowledge Too close to call

Gemma 3 1B: 7.0 (#314), Llama 2-70B: 7.4 (#310)

BenchmarkGemma 3 1BLlama 2-70B
GPQA Diamond19.9%26.3%
LMArena Expert—1039
ARC (AI2) Challenge—78.3%
BoolQ—88.6%
MMLU—69.9%
OpenBookQA—60.2%
TriviaQA—87.6%

Multilingual Not comparable

Gemma 3 1B: —, Llama 2-70B: 27.7 (#274)

BenchmarkGemma 3 1BLlama 2-70B
LMArena Non-English—1045
LMArena Chinese—995
LMArena French—1090
LMArena German—1041
LMArena Japanese—927
LMArena Korean—964
LMArena Russian—1083
LMArena Spanish—1143

Instruction Following Not comparable

Gemma 3 1B: —, Llama 2-70B: 54.9 (#278)

BenchmarkGemma 3 1BLlama 2-70B
LMArena Instruction Following—1071

Long Context Not comparable

Gemma 3 1B: —, Llama 2-70B: 32.3 (#270)

BenchmarkGemma 3 1BLlama 2-70B
LMArena Longer Query—1062

Writing & Preference Not comparable

Gemma 3 1B: —, Llama 2-70B: 32.3 (#279)

BenchmarkGemma 3 1BLlama 2-70B
LMArena Text—1115
LMArena Creative Writing—1075
LMArena Multi-Turn—1088

Frequently asked questions

Is Gemma 3 1B better than Llama 2-70B?

Llama 2-70B is the stronger model overall, scoring 24.4 to 21.1 on the Noometry Index.

How many benchmarks do Gemma 3 1B and Llama 2-70B share?

2 benchmarks have published results for both models. Gemma 3 1B has 4 scored results on Noometry and Llama 2-70B has 35.

Related comparisons

Go deeper