Noometry

Model comparison

Claude Fable 5.1 vs Gemma 7B

Claude Fable 5.1 is the stronger model overall, scoring 69.0 to 30.0 on the Noometry Index.

Last verified . 14 shared benchmarks.

Claude Fable 5.1 Anthropic

69.0

Rank #2 Confirmed

Gemma 7B Google

30.0

Rank #299 Confirmed

Summary

  • They share 14 benchmarks with published results for both. Claude Fable 5.1 scores higher in 8 categories and Gemma 7B in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Claude Fable 5.1 leads 89.6 to 31.2.
  • Gemma 7B has downloadable open weights; the other is API-only.

Side by side

Claude Fable 5.1Gemma 7B
ProviderAnthropicGoogle
Noometry Index69.030.0
Released2026-09-012024-02-21
WeightsProprietaryOpen
Context window1M—
Max output128K—
Input $ / M tokens$10—
Output $ / M tokens$50—
Results tracked5227

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Claude Fable 5.1 leads

Claude Fable 5.1: 74.7 (#1), Gemma 7B: 30.5 (#294)

BenchmarkClaude Fable 5.1Gemma 7B
LMArena Coding15281048
FrontierCode50.9%—
CursorBench51.8%—
LMArena WebDev1744—
FrontierSWE56.3%—
SciCode63.1%—
GSO88.2%—
WeirdML92.9%—
MirrorCode73.3%—
ALE-Bench2,143—
HumanEval+—28.7%
MBPP+—43.4%

Agentic & Tool Use Not comparable

Claude Fable 5.1: 50.7 (#5), Gemma 7B: —

BenchmarkClaude Fable 5.1Gemma 7B
APEX-Agents68.6%—
Remote Labor Index17.9%—
GDP.pdf29.6%—
Vending-Bench 25,422—

Reasoning Claude Fable 5.1 leads

Claude Fable 5.1: 76.7 (#7), Gemma 7B: 19.9 (#249)

BenchmarkClaude Fable 5.1Gemma 7B
LMArena Hard Prompts15261042
Epoch Capabilities Index164.7111.99
ARC-AGI-290%—
NYT Connections (extended)90%—
ARC-AGI-197.5%—
CritPt31.1%—
Chess Puzzles47%—
EBR-Bench57.1%—
Mystery Game Puzzles58%—
DTBench97.6%—
LMCA65.5%—
Adversarial NLI—48.7%
BIG-Bench Hard—55.1%
HellaSwag—82.2%
PIQA—81.2%
WinoGrande—79%

Math Claude Fable 5.1 leads

Claude Fable 5.1: 89.6 (#4), Gemma 7B: 31.2 (#228)

BenchmarkClaude Fable 5.1Gemma 7B
LMArena Math15251066
FrontierMath (Tiers 1-3)90.2%—
FrontierMath Tier 487.8%—
OTIS Mock AIME 2024-2025100%—
ProofBench100%—
FrontierMath Erdős0%—
GSM8K—46.4%

Knowledge Claude Fable 5.1 leads

Claude Fable 5.1: 69.6 (#6), Gemma 7B: 27.3 (#252)

BenchmarkClaude Fable 5.1Gemma 7B
LMArena Expert15351001
Humanity's Last Exam46.5%—
SimpleQA Verified70.8%—
ARC (AI2) Challenge—78.3%
BoolQ—83.2%
MMLU—66.1%
OpenBookQA—78.6%
TriviaQA—72.3%

Multimodal Not comparable

Claude Fable 5.1: 53.9 (#4), Gemma 7B: —

BenchmarkClaude Fable 5.1Gemma 7B
LMArena Vision1318—
Blueprint-Bench 241.9%—
Furniture Assembly70%—
LMArena Document1513—

Multilingual Claude Fable 5.1 leads

Claude Fable 5.1: 59.1 (#3), Gemma 7B: 25.1 (#287)

BenchmarkClaude Fable 5.1Gemma 7B
LMArena Non-English1507999
LMArena Chinese15861035
LMArena French15251025
LMArena Russian1521993
LMArena German1500—
LMArena Japanese1543—
LMArena Korean1534—
LMArena Spanish1516—

Instruction Following Claude Fable 5.1 leads

Claude Fable 5.1: 79.2 (#6), Gemma 7B: 51.5 (#295)

BenchmarkClaude Fable 5.1Gemma 7B
LMArena Instruction Following15171017

Long Context Claude Fable 5.1 leads

Claude Fable 5.1: 46.7 (#20), Gemma 7B: 31.1 (#282)

BenchmarkClaude Fable 5.1Gemma 7B
LMArena Longer Query15221022

Writing & Preference Claude Fable 5.1 leads

Claude Fable 5.1: 79.2 (#2), Gemma 7B: 27.1 (#302)

BenchmarkClaude Fable 5.1Gemma 7B
LMArena Text15101056
LMArena Creative Writing15071024
LMArena Multi-Turn1492963
EQ-Bench Creative Writing2162—

Frequently asked questions

Is Claude Fable 5.1 better than Gemma 7B?

Claude Fable 5.1 is the stronger model overall, scoring 69.0 to 30.0 on the Noometry Index.

Is Claude Fable 5.1 or Gemma 7B better for coding?

Claude Fable 5.1 scores higher on coding benchmarks: 74.7 versus 30.5 in the Noometry coding category.

How many benchmarks do Claude Fable 5.1 and Gemma 7B share?

14 benchmarks have published results for both models. Claude Fable 5.1 has 52 scored results on Noometry and Gemma 7B has 27.

Related comparisons

Go deeper