Noometry

Model comparison

Llama 3-70B vs Qwen2.5-Coder-32B

Qwen2.5-Coder-32B is the stronger model overall, scoring 32.9 to 28.5 on the Noometry Index.

Last verified . 19 shared benchmarks.

Llama 3-70B Meta

28.5

Rank #317 Confirmed

Qwen2.5-Coder-32B Alibaba (Qwen)

32.9

Rank #239 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Llama 3-70B scores higher in 3 categories and Qwen2.5-Coder-32B in 5 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Qwen2.5-Coder-32B leads 30.8 to 12.0.
  • The biggest single-benchmark swing is BigCodeBench Instruct: 43.6% for Llama 3-70B and 49% for Qwen2.5-Coder-32B.

Side by side

Llama 3-70BQwen2.5-Coder-32B
ProviderMetaAlibaba (Qwen)
Noometry Index28.532.9
Released2024-04-182024-09-18
WeightsOpenOpen
Context window—33K
Max output—29K
Input $ / M tokens—$0.66
Output $ / M tokens—$1
Results tracked3131

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 3-70B leads

Llama 3-70B: 35.3 (#215), Qwen2.5-Coder-32B: 22.4 (#328)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
BigCodeBench Instruct43.6%49%
LMArena Coding12061276
BigCodeBench Complete54.5%58%
HumanEval+72%87.2%
MBPP+69%77%
SWE-bench Verified (bash only)—9%
Aider Polyglot—16.4%
LiveBench Coding—56.9%

Agentic & Tool Use Not comparable

Llama 3-70B: 20.9 (#132), Qwen2.5-Coder-32B: —

BenchmarkLlama 3-70BQwen2.5-Coder-32B
Cybench5%—

Reasoning Qwen2.5-Coder-32B leads

Llama 3-70B: 17.4 (#284), Qwen2.5-Coder-32B: 20.4 (#220)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
LMArena Hard Prompts11951251
Epoch Capabilities Index122.91119.41
WinoGrande83.5%80.8%
Kagi LLM Benchmark35.1%—
LiveBench Reasoning—42.1%
DTBench54.2%—
LiveBench Data Analysis—49.9%
ForecastBench57.1—
HellaSwag—83%
LiveBench—46.2%

Math Qwen2.5-Coder-32B leads

Llama 3-70B: 12.0 (#299), Qwen2.5-Coder-32B: 30.8 (#203)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
LMArena Math12181251
OTIS Mock AIME 2024-20254.3%—
LiveBench Math—46.6%
MATH Level 522.6%—
GSM8K—93%

Knowledge Qwen2.5-Coder-32B leads

Llama 3-70B: 20.8 (#272), Qwen2.5-Coder-32B: 32.9 (#200)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
LMArena Expert11491221
MMLU79.3%79.1%
GPQA Diamond40.6%—
ARC (AI2) Challenge—70.5%

Multilingual Qwen2.5-Coder-32B leads

Llama 3-70B: 33.8 (#246), Qwen2.5-Coder-32B: 38.0 (#230)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
LMArena Non-English11421205
LMArena Chinese11141222
LMArena Russian11591228
LMArena French1231—
LMArena German1169—
LMArena Japanese1017—
LMArena Korean1017—
LMArena Spanish1241—

Instruction Following Llama 3-70B leads

Llama 3-70B: 62.8 (#233), Qwen2.5-Coder-32B: 61.6 (#240)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
LMArena Instruction Following11941223
LiveBench Instruction Following—58.7%

Long Context Qwen2.5-Coder-32B leads

Llama 3-70B: 35.6 (#235), Qwen2.5-Coder-32B: 37.9 (#204)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
LMArena Longer Query11741251

Writing & Preference Llama 3-70B leads

Llama 3-70B: 43.0 (#225), Qwen2.5-Coder-32B: 41.8 (#235)

BenchmarkLlama 3-70BQwen2.5-Coder-32B
LMArena Text12211230
LMArena Creative Writing12101174
LMArena Multi-Turn12231222
LiveBench Language—23.3%

Frequently asked questions

Is Llama 3-70B better than Qwen2.5-Coder-32B?

Qwen2.5-Coder-32B is the stronger model overall, scoring 32.9 to 28.5 on the Noometry Index.

Is Llama 3-70B or Qwen2.5-Coder-32B better for coding?

Llama 3-70B scores higher on coding benchmarks: 35.3 versus 22.4 in the Noometry coding category.

How many benchmarks do Llama 3-70B and Qwen2.5-Coder-32B share?

19 benchmarks have published results for both models. Llama 3-70B has 31 scored results on Noometry and Qwen2.5-Coder-32B has 31.

Related comparisons

Go deeper