Noometry

Model comparison

Claude 3 Sonnet vs Qwen2.5 32B Instruct

Qwen2.5 32B Instruct is the stronger model overall, scoring 30.1 to 29.0 on the Noometry Index.

Last verified . 6 shared benchmarks.

Claude 3 Sonnet Anthropic

29.0

Rank #319 Confirmed

Qwen2.5 32B Instruct Alibaba (Qwen)

30.1

Rank #297 Confirmed

Summary

  • They share 6 benchmarks with published results for both. Claude 3 Sonnet scores higher in 1 category and Qwen2.5 32B Instruct in 3 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in coding, where Qwen2.5 32B Instruct leads 38.7 to 29.6.
  • The biggest single-benchmark swing is MATH Level 5: 18.2% for Claude 3 Sonnet and 56.1% for Qwen2.5 32B Instruct.
  • Qwen2.5 32B Instruct has downloadable open weights; the other is API-only.

Side by side

Claude 3 SonnetQwen2.5 32B Instruct
ProviderAnthropicAlibaba (Qwen)
Noometry Index29.030.1
Released2024-02-292024-09
WeightsProprietaryOpen
Context window—131K
Max output—8K
Input $ / M tokens—$0.70
Output $ / M tokens—$2.80
Results tracked307

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Qwen2.5 32B Instruct leads

Claude 3 Sonnet: 29.6 (#302), Qwen2.5 32B Instruct: 38.7 (#169)

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
BigCodeBench Instruct42.7%45%
BigCodeBench Complete53.8%52.3%
WeirdML10.2%—
LMArena Coding1223—
HumanEval+64%—
MBPP+69.3%—

Reasoning Claude 3 Sonnet leads

Claude 3 Sonnet: 20.5 (#237), Qwen2.5 32B Instruct: 19.2 (#266)

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
Epoch Capabilities Index120.7128.52
Chess Puzzles—0%
LMArena Hard Prompts1197—
DTBench53.6%—
WinoGrande75.1%—

Math Qwen2.5 32B Instruct leads

Claude 3 Sonnet: 10.7 (#310), Qwen2.5 32B Instruct: 16.2 (#296)

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
OTIS Mock AIME 2024-20252.5%7.4%
MATH Level 518.2%56.1%
LMArena Math1213—

Knowledge Qwen2.5 32B Instruct leads

Claude 3 Sonnet: 21.1 (#276), Qwen2.5 32B Instruct: 24.9 (#266)

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
GPQA Diamond40.6%46.1%
LMArena Expert1173—
MMLU75.9%—

Multimodal Not comparable

Claude 3 Sonnet: 25.2 (#125), Qwen2.5 32B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
LMArena Vision984—

Multilingual Not comparable

Claude 3 Sonnet: 37.8 (#234), Qwen2.5 32B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
LMArena Non-English1205—
LMArena Chinese1189—
LMArena French1229—
LMArena German1204—
LMArena Japanese1131—
LMArena Korean1128—
LMArena Russian1227—
LMArena Spanish1204—

Instruction Following Not comparable

Claude 3 Sonnet: 62.8 (#235), Qwen2.5 32B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
LMArena Instruction Following1199—

Long Context Not comparable

Claude 3 Sonnet: 36.7 (#228), Qwen2.5 32B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
LMArena Longer Query1211—

Writing & Preference Not comparable

Claude 3 Sonnet: 42.1 (#238), Qwen2.5 32B Instruct: —

BenchmarkClaude 3 SonnetQwen2.5 32B Instruct
LMArena Text1218—
LMArena Creative Writing1186—
LMArena Multi-Turn1227—

Frequently asked questions

Is Claude 3 Sonnet better than Qwen2.5 32B Instruct?

Qwen2.5 32B Instruct is the stronger model overall, scoring 30.1 to 29.0 on the Noometry Index.

Is Claude 3 Sonnet or Qwen2.5 32B Instruct better for coding?

Qwen2.5 32B Instruct scores higher on coding benchmarks: 38.7 versus 29.6 in the Noometry coding category.

How many benchmarks do Claude 3 Sonnet and Qwen2.5 32B Instruct share?

6 benchmarks have published results for both models. Claude 3 Sonnet has 30 scored results on Noometry and Qwen2.5 32B Instruct has 7.

Related comparisons

Go deeper