Noometry

Model comparison

Claude 3 Sonnet vs Llama 3.2 3B

Claude 3 Sonnet and Llama 3.2 3B score almost the same on the Noometry Index (29.0 vs 28.9), so choose on price, context window or the category you care about most.

Last verified . 15 shared benchmarks.

Claude 3 Sonnet Anthropic

29.0

Rank #319 Confirmed

Llama 3.2 3B Meta

28.9

Rank #321 Confirmed

Summary

  • They share 15 benchmarks with published results for both. Claude 3 Sonnet scores higher in 5 categories and Llama 3.2 3B in 3 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Llama 3.2 3B leads 32.4 to 10.7.
  • The biggest single-benchmark swing is BigCodeBench Complete: 53.8% for Claude 3 Sonnet and 28.3% for Llama 3.2 3B.
  • Llama 3.2 3B has downloadable open weights; the other is API-only.

Side by side

Claude 3 SonnetLlama 3.2 3B
ProviderAnthropicMeta
Noometry Index29.028.9
Released2024-02-292024-09-24
WeightsProprietaryOpen
Context window—131K
Max output—118K
Input $ / M tokens—$0.05
Output $ / M tokens—$0.33
Results tracked3018

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude 3 Sonnet leads

Claude 3 Sonnet: 29.6 (#302), Llama 3.2 3B: 27.6 (#319)

BenchmarkClaude 3 SonnetLlama 3.2 3B
BigCodeBench Instruct42.7%23.4%
LMArena Coding12231098
BigCodeBench Complete53.8%28.3%
WeirdML10.2%—
HumanEval+64%—
MBPP+69.3%—

Agentic & Tool Use Not comparable

Claude 3 Sonnet: —, Llama 3.2 3B: 20.1 (#143)

BenchmarkClaude 3 SonnetLlama 3.2 3B
Berkeley Function Calling Leaderboard—21.9%
BALROG—10.1%

Reasoning Too close to call

Claude 3 Sonnet: 20.5 (#237), Llama 3.2 3B: 21.0 (#228)

BenchmarkClaude 3 SonnetLlama 3.2 3B
LMArena Hard Prompts11971095
DTBench53.6%—
Epoch Capabilities Index120.7—
WinoGrande75.1%—

Math Llama 3.2 3B leads

Claude 3 Sonnet: 10.7 (#310), Llama 3.2 3B: 32.4 (#214)

BenchmarkClaude 3 SonnetLlama 3.2 3B
LMArena Math12131126
OTIS Mock AIME 2024-20252.5%—
MATH Level 518.2%—

Knowledge Llama 3.2 3B leads

Claude 3 Sonnet: 21.1 (#276), Llama 3.2 3B: 29.7 (#235)

BenchmarkClaude 3 SonnetLlama 3.2 3B
LMArena Expert11731090
GPQA Diamond40.6%—
MMLU75.9%—

Multimodal Not comparable

Claude 3 Sonnet: 25.2 (#125), Llama 3.2 3B: —

BenchmarkClaude 3 SonnetLlama 3.2 3B
LMArena Vision984—

Multilingual Claude 3 Sonnet leads

Claude 3 Sonnet: 37.8 (#234), Llama 3.2 3B: 26.2 (#281)

BenchmarkClaude 3 SonnetLlama 3.2 3B
LMArena Non-English12051019
LMArena Chinese11891017
LMArena German12041056
LMArena Russian1227949
LMArena French1229—
LMArena Japanese1131—
LMArena Korean1128—
LMArena Spanish1204—

Instruction Following Claude 3 Sonnet leads

Claude 3 Sonnet: 62.8 (#235), Llama 3.2 3B: 56.0 (#275)

BenchmarkClaude 3 SonnetLlama 3.2 3B
LMArena Instruction Following11991089

Long Context Claude 3 Sonnet leads

Claude 3 Sonnet: 36.7 (#228), Llama 3.2 3B: 33.4 (#261)

BenchmarkClaude 3 SonnetLlama 3.2 3B
LMArena Longer Query12111100

Writing & Preference Claude 3 Sonnet leads

Claude 3 Sonnet: 42.1 (#238), Llama 3.2 3B: 24.7 (#307)

BenchmarkClaude 3 SonnetLlama 3.2 3B
LMArena Text12181110
LMArena Creative Writing11861094
LMArena Multi-Turn12271105
EQ-Bench Creative Writing—595

Frequently asked questions

Is Claude 3 Sonnet better than Llama 3.2 3B?

Claude 3 Sonnet and Llama 3.2 3B score almost the same on the Noometry Index (29.0 vs 28.9), so choose on price, context window or the category you care about most.

Is Claude 3 Sonnet or Llama 3.2 3B better for coding?

Claude 3 Sonnet scores higher on coding benchmarks: 29.6 versus 27.6 in the Noometry coding category.

How many benchmarks do Claude 3 Sonnet and Llama 3.2 3B share?

15 benchmarks have published results for both models. Claude 3 Sonnet has 30 scored results on Noometry and Llama 3.2 3B has 18.

Related comparisons

Go deeper