Noometry

Model comparison

Claude 3.5 Haiku vs Grok 3

Grok 3 is the stronger model overall, scoring 39.9 to 29.2 on the Noometry Index.

Last verified . 33 shared benchmarks.

Claude 3.5 Haiku Anthropic

29.2

Rank #315 Confirmed

Grok 3 xAI

39.9

Rank #157 Confirmed

Summary

  • They share 33 benchmarks with published results for both. Claude 3.5 Haiku scores higher in 1 category and Grok 3 in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Grok 3 leads 46.2 to 18.7.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 4.3% for Claude 3.5 Haiku and 55.6% for Grok 3.

Side by side

Claude 3.5 HaikuGrok 3
ProviderAnthropicxAI
Noometry Index29.239.9
Released2024-10-222025-04-09
WeightsProprietaryProprietary
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked4940

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok 3 leads

Claude 3.5 Haiku: 32.9 (#265), Grok 3: 41.9 (#115)

BenchmarkClaude 3.5 HaikuGrok 3
Aider Polyglot28%53.3%
WeirdML30.7%37.2%
LMArena Coding12861432
SciCode27.4%—
BigCodeBench Instruct46.1%—
LiveBench Coding51.4%—
BigCodeBench Complete59%—
CadEval32%—

Agentic & Tool Use Grok 3 leads

Claude 3.5 Haiku: 28.0 (#95), Grok 3: 30.5 (#76)

BenchmarkClaude 3.5 HaikuGrok 3
BALROG19.3%29.5%

Reasoning Claude 3.5 Haiku leads

Claude 3.5 Haiku: 17.7 (#290), Grok 3: 13.7 (#333)

BenchmarkClaude 3.5 HaikuGrok 3
LMArena Hard Prompts12511434
Epoch Capabilities Index127.15138.33
ARC-AGI-2—0%
SimpleBench—36.1%
Kagi LLM Benchmark—61.3%
ARC-AGI-1—5.5%
CritPt0%—
LiveBench Reasoning28.1%—
DTBench56.7%—
LiveBench Data Analysis48.5%—
LiveBench43.5%—

Math Grok 3 leads

Claude 3.5 Haiku: 14.7 (#300), Grok 3: 38.0 (#145)

BenchmarkClaude 3.5 HaikuGrok 3
OTIS Mock AIME 2024-20254.3%55.6%
Omni-MATH22.4%46.4%
LMArena Math12441391
MATH Level 546.4%88.7%
FrontierMath (Feb 2025 set)0.3%3.8%
LiveBench Math35.5%—
FrontierMath Tier 4 (v1)—0%

Knowledge Grok 3 leads

Claude 3.5 Haiku: 18.7 (#281), Grok 3: 46.2 (#82)

BenchmarkClaude 3.5 HaikuGrok 3
GPQA Diamond38.1%75.8%
MMLU-Pro60.5%78.8%
Confabulations36.7%14.2%
GPQA (HELM)36.3%65%
LMArena Expert12081421
Vectara Hallucination Rate—5.8%
MMLU74.3%—

Multimodal Not comparable

Claude 3.5 Haiku: 26.8 (#117), Grok 3: —

BenchmarkClaude 3.5 HaikuGrok 3
LMArena Vision1092—
GeoBench34%—

Multilingual Grok 3 leads

Claude 3.5 Haiku: 40.0 (#218), Grok 3: 52.3 (#87)

BenchmarkClaude 3.5 HaikuGrok 3
LMArena Non-English12381410
LMArena Chinese12291448
LMArena French12641460
LMArena German12371431
LMArena Japanese11751387
LMArena Korean11731373
LMArena Russian12531416
LMArena Spanish12611417

Instruction Following Grok 3 leads

Claude 3.5 Haiku: 62.9 (#234), Grok 3: 75.0 (#73)

BenchmarkClaude 3.5 HaikuGrok 3
IFEval79.2%88.4%
LMArena Instruction Following12411409
LiveBench Instruction Following61.9%—

Long Context Too close to call

Claude 3.5 Haiku: 38.3 (#200), Grok 3: 38.7 (#192)

BenchmarkClaude 3.5 HaikuGrok 3
LMArena Longer Query12611439
Fiction.LiveBench—58.3%

Writing & Preference Grok 3 leads

Claude 3.5 Haiku: 42.7 (#234), Grok 3: 55.8 (#141)

BenchmarkClaude 3.5 HaikuGrok 3
LMArena Text12551426
LMArena Creative Writing12331414
Short-Story Creative Writing73.5%76.4%
EQ-Bench Creative Writing11461186
WildBench76%84.9%
LMArena Multi-Turn12651425
LiveBench Language35.4%—

Frequently asked questions

Is Claude 3.5 Haiku better than Grok 3?

Grok 3 is the stronger model overall, scoring 39.9 to 29.2 on the Noometry Index.

Is Claude 3.5 Haiku or Grok 3 better for coding?

Grok 3 scores higher on coding benchmarks: 41.9 versus 32.9 in the Noometry coding category.

How many benchmarks do Claude 3.5 Haiku and Grok 3 share?

33 benchmarks have published results for both models. Claude 3.5 Haiku has 49 scored results on Noometry and Grok 3 has 40.

Related comparisons

Go deeper