Noometry

Model comparison

DeepSeek-R1-Distill-Qwen-1.5B vs Llama 3-8B

DeepSeek-R1-Distill-Qwen-1.5B and Llama 3-8B score almost the same on the Noometry Index (26.1 vs 25.5), so choose on price, context window or the category you care about most.

Last verified . 5 shared benchmarks.

DeepSeek-R1-Distill-Qwen-1.5B DeepSeek

26.1

Rank #339 Confirmed

Llama 3-8B Meta

25.5

Rank #344 Confirmed

Summary

  • They share 5 benchmarks with published results for both. DeepSeek-R1-Distill-Qwen-1.5B scores higher in 3 categories and Llama 3-8B in 1 category; 4 gaps are clear of the uncertainty.
  • The widest gap is in math, where DeepSeek-R1-Distill-Qwen-1.5B leads 23.0 to 8.8.
  • The biggest single-benchmark swing is BigCodeBench Complete: 7.9% for DeepSeek-R1-Distill-Qwen-1.5B and 36.9% for Llama 3-8B.

Side by side

DeepSeek-R1-Distill-Qwen-1.5BLlama 3-8B
ProviderDeepSeekMeta
Noometry Index26.125.5
Released2025-01-202024-04-18
WeightsOpenOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked534

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 3-8B leads

DeepSeek-R1-Distill-Qwen-1.5B: 21.8 (#336), Llama 3-8B: 31.0 (#289)

BenchmarkDeepSeek-R1-Distill-Qwen-1.5BLlama 3-8B
BigCodeBench Instruct7%31.9%
BigCodeBench Complete7.9%36.9%
LMArena Coding—1152
HumanEval+—56.7%
MBPP+—54.8%

Reasoning DeepSeek-R1-Distill-Qwen-1.5B leads

DeepSeek-R1-Distill-Qwen-1.5B: 19.2 (#262), Llama 3-8B: 14.3 (#326)

BenchmarkDeepSeek-R1-Distill-Qwen-1.5BLlama 3-8B
Chess Puzzles0%0%
LMArena Hard Prompts—1133
DTBench—43.9%
Adversarial NLI—57.3%
Epoch Capabilities Index—116.45
ForecastBench—58.6
WinoGrande—75.7%

Math DeepSeek-R1-Distill-Qwen-1.5B leads

DeepSeek-R1-Distill-Qwen-1.5B: 23.0 (#274), Llama 3-8B: 8.8 (#323)

BenchmarkDeepSeek-R1-Distill-Qwen-1.5BLlama 3-8B
OTIS Mock AIME 2024-202521.4%1.9%
LMArena Math—1151
MATH Level 5—6.1%

Knowledge DeepSeek-R1-Distill-Qwen-1.5B leads

DeepSeek-R1-Distill-Qwen-1.5B: 16.0 (#290), Llama 3-8B: 7.8 (#308)

BenchmarkDeepSeek-R1-Distill-Qwen-1.5BLlama 3-8B
GPQA Diamond33.6%26.1%
LMArena Expert—1113
ARC (AI2) Challenge—82.8%
MMLU—68.8%
OpenBookQA—82.6%
TriviaQA—67.7%

Multilingual Not comparable

DeepSeek-R1-Distill-Qwen-1.5B: —, Llama 3-8B: 30.8 (#261)

BenchmarkDeepSeek-R1-Distill-Qwen-1.5BLlama 3-8B
LMArena Non-English—1098
LMArena Chinese—1076
LMArena French—1159
LMArena German—1104
LMArena Japanese—967
LMArena Korean—1004
LMArena Russian—1109
LMArena Spanish—1173

Instruction Following Not comparable

DeepSeek-R1-Distill-Qwen-1.5B: —, Llama 3-8B: 58.4 (#260)

BenchmarkDeepSeek-R1-Distill-Qwen-1.5BLlama 3-8B
LMArena Instruction Following—1127

Long Context Not comparable

DeepSeek-R1-Distill-Qwen-1.5B: —, Llama 3-8B: 34.2 (#251)

BenchmarkDeepSeek-R1-Distill-Qwen-1.5BLlama 3-8B
LMArena Longer Query—1128

Writing & Preference Not comparable

DeepSeek-R1-Distill-Qwen-1.5B: —, Llama 3-8B: 37.5 (#256)

BenchmarkDeepSeek-R1-Distill-Qwen-1.5BLlama 3-8B
LMArena Text—1166
LMArena Creative Writing—1150
LMArena Multi-Turn—1152

Frequently asked questions

Is DeepSeek-R1-Distill-Qwen-1.5B better than Llama 3-8B?

DeepSeek-R1-Distill-Qwen-1.5B and Llama 3-8B score almost the same on the Noometry Index (26.1 vs 25.5), so choose on price, context window or the category you care about most.

Is DeepSeek-R1-Distill-Qwen-1.5B or Llama 3-8B better for coding?

Llama 3-8B scores higher on coding benchmarks: 31.0 versus 21.8 in the Noometry coding category.

How many benchmarks do DeepSeek-R1-Distill-Qwen-1.5B and Llama 3-8B share?

5 benchmarks have published results for both models. DeepSeek-R1-Distill-Qwen-1.5B has 5 scored results on Noometry and Llama 3-8B has 34.

Related comparisons

Go deeper