Noometry

Model comparison

Claude Fable 5.1 vs Llama 3-8B

Claude Fable 5.1 is the stronger model overall, scoring 69.0 to 25.5 on the Noometry Index.

Last verified . 21 shared benchmarks.

Claude Fable 5.1 Anthropic

69.0

Rank #2 Confirmed

Llama 3-8B Meta

25.5

Rank #344 Confirmed

Summary

  • They share 21 benchmarks with published results for both. Claude Fable 5.1 scores higher in 8 categories and Llama 3-8B in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Claude Fable 5.1 leads 89.6 to 8.8.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 100% for Claude Fable 5.1 and 1.9% for Llama 3-8B.
  • Llama 3-8B has downloadable open weights; the other is API-only.

Side by side

Claude Fable 5.1Llama 3-8B
ProviderAnthropicMeta
Noometry Index69.025.5
Released2026-09-012024-04-18
WeightsProprietaryOpen
Context window1M—
Max output128K—
Input $ / M tokens$10—
Output $ / M tokens$50—
Results tracked5234

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude Fable 5.1 leads

Claude Fable 5.1: 74.7 (#1), Llama 3-8B: 31.0 (#289)

BenchmarkClaude Fable 5.1Llama 3-8B
LMArena Coding15281152
FrontierCode50.9%—
CursorBench51.8%—
LMArena WebDev1744—
FrontierSWE56.3%—
SciCode63.1%—
GSO88.2%—
WeirdML92.9%—
BigCodeBench Instruct—31.9%
MirrorCode73.3%—
BigCodeBench Complete—36.9%
ALE-Bench2,143—
HumanEval+—56.7%
MBPP+—54.8%

Agentic & Tool Use Not comparable

Claude Fable 5.1: 50.7 (#5), Llama 3-8B: —

BenchmarkClaude Fable 5.1Llama 3-8B
APEX-Agents68.6%—
Remote Labor Index17.9%—
GDP.pdf29.6%—
Vending-Bench 25,422—

Reasoning Claude Fable 5.1 leads

Claude Fable 5.1: 76.7 (#7), Llama 3-8B: 14.3 (#326)

BenchmarkClaude Fable 5.1Llama 3-8B
Chess Puzzles47%0%
LMArena Hard Prompts15261133
DTBench97.6%43.9%
Epoch Capabilities Index164.7116.45
ARC-AGI-290%—
NYT Connections (extended)90%—
ARC-AGI-197.5%—
CritPt31.1%—
EBR-Bench57.1%—
Mystery Game Puzzles58%—
LMCA65.5%—
Adversarial NLI—57.3%
ForecastBench—58.6
WinoGrande—75.7%

Math Claude Fable 5.1 leads

Claude Fable 5.1: 89.6 (#4), Llama 3-8B: 8.8 (#323)

BenchmarkClaude Fable 5.1Llama 3-8B
OTIS Mock AIME 2024-2025100%1.9%
LMArena Math15251151
FrontierMath (Tiers 1-3)90.2%—
FrontierMath Tier 487.8%—
ProofBench100%—
MATH Level 5—6.1%
FrontierMath Erdős0%—

Knowledge Claude Fable 5.1 leads

Claude Fable 5.1: 69.6 (#6), Llama 3-8B: 7.8 (#308)

BenchmarkClaude Fable 5.1Llama 3-8B
LMArena Expert15351113
GPQA Diamond—26.1%
Humanity's Last Exam46.5%—
SimpleQA Verified70.8%—
ARC (AI2) Challenge—82.8%
MMLU—68.8%
OpenBookQA—82.6%
TriviaQA—67.7%

Multimodal Not comparable

Claude Fable 5.1: 53.9 (#4), Llama 3-8B: —

BenchmarkClaude Fable 5.1Llama 3-8B
LMArena Vision1318—
Blueprint-Bench 241.9%—
Furniture Assembly70%—
LMArena Document1513—

Multilingual Claude Fable 5.1 leads

Claude Fable 5.1: 59.1 (#3), Llama 3-8B: 30.8 (#261)

BenchmarkClaude Fable 5.1Llama 3-8B
LMArena Non-English15071098
LMArena Chinese15861076
LMArena French15251159
LMArena German15001104
LMArena Japanese1543967
LMArena Korean15341004
LMArena Russian15211109
LMArena Spanish15161173

Instruction Following Claude Fable 5.1 leads

Claude Fable 5.1: 79.2 (#6), Llama 3-8B: 58.4 (#260)

BenchmarkClaude Fable 5.1Llama 3-8B
LMArena Instruction Following15171127

Long Context Claude Fable 5.1 leads

Claude Fable 5.1: 46.7 (#20), Llama 3-8B: 34.2 (#251)

BenchmarkClaude Fable 5.1Llama 3-8B
LMArena Longer Query15221128

Writing & Preference Claude Fable 5.1 leads

Claude Fable 5.1: 79.2 (#2), Llama 3-8B: 37.5 (#256)

BenchmarkClaude Fable 5.1Llama 3-8B
LMArena Text15101166
LMArena Creative Writing15071150
LMArena Multi-Turn14921152
EQ-Bench Creative Writing2162—

Frequently asked questions

Is Claude Fable 5.1 better than Llama 3-8B?

Claude Fable 5.1 is the stronger model overall, scoring 69.0 to 25.5 on the Noometry Index.

Is Claude Fable 5.1 or Llama 3-8B better for coding?

Claude Fable 5.1 scores higher on coding benchmarks: 74.7 versus 31.0 in the Noometry coding category.

How many benchmarks do Claude Fable 5.1 and Llama 3-8B share?

21 benchmarks have published results for both models. Claude Fable 5.1 has 52 scored results on Noometry and Llama 3-8B has 34.

Related comparisons

Go deeper