Model comparison

Granite 4.1 8b vs o3

o3 is the stronger model overall, scoring 47.4 to 36.8 on the Noometry Index.

Last verified October 10, 2026. 12 shared benchmarks.

Granite 4.1 8b IBM

36.8

Rank #202 Confirmed

o3 OpenAI

47.4

Rank #53 Confirmed

Summary

They share 12 benchmarks with published results for both. Granite 4.1 8b scores higher in 0 categories and o3 in 8 categories; 8 gaps are clear of the uncertainty.
The widest gap is in knowledge, where o3 leads 54.5 to 35.5.
Granite 4.1 8b has downloadable open weights; the other is API-only.

Side by side

Granite 4.1 8b and o3 specifications
	Granite 4.1 8b	o3
Provider	IBM	OpenAI
Noometry Index	36.8	47.4
Released	—	2025-04-16
Weights	Open	Proprietary
Context window	—	200K
Max output	—	100K
Input $ / M tokens	—	$2
Output $ / M tokens	—	$8
Results tracked	13	63

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3 leads

Granite 4.1 8b: 29.9 (#291), o3: 47.3 (#57)

Coding benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Coding	1313	1408
SWE-bench Verified	—	62.3%
SWE-bench Verified (bash only)	—	58.4%
Aider Polyglot	—	81.3%
LMArena WebDev	1190	—
GSO	—	8.8%
WeirdML	—	52.4%
CadEval	—	74%
ALE-Bench	—	933.55

Agentic & Tool Use Not comparable

Granite 4.1 8b: —, o3: 34.3 (#42)

Agentic & Tool Use benchmarks
Benchmark	Granite 4.1 8b	o3
Berkeley Function Calling Leaderboard	—	63%
GDPval	—	30.8%
DeepResearch Bench	—	45.2%
OSWorld	—	23%
LMArena Search	—	1144
METR Time Horizons	—	65.4%

Reasoning o3 leads

Granite 4.1 8b: 24.6 (#142), o3: 32.1 (#66)

Reasoning benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Hard Prompts	1294	1402
ARC-AGI-2	—	6.5%
SimpleBench	—	53.1%
Kagi LLM Benchmark	—	67.6%
ARC-AGI-1	—	60.8%
CritPt	—	1.4%
Chess Puzzles	—	38%
EnigmaEval	—	13.1%
Mystery Game Puzzles	—	29%
DTBench	—	84.8%
LMCA	—	39.7%
Epoch Capabilities Index	—	146.88
ForecastBench	—	62.5

Math o3 leads

Granite 4.1 8b: 33.8 (#167), o3: 48.9 (#49)

Math benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Math	1311	1426
FrontierMath (Tiers 1-3)	—	33.3%
OTIS Mock AIME 2024-2025	—	84.4%
Omni-MATH	—	71.4%
MATH Level 5	—	97.8%
FrontierMath (Feb 2025 set)	—	18.7%
FrontierMath Tier 4 (v1)	—	2.1%

Knowledge o3 leads

Granite 4.1 8b: 35.5 (#171), o3: 54.5 (#47)

Knowledge benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Expert	1310	1403
GPQA Diamond	—	81.8%
Humanity's Last Exam	—	20.3%
SimpleQA Verified	—	49.4%
MMLU-Pro	—	85.9%
Confabulations	—	14.4%
GPQA (HELM)	—	75.3%

Multimodal Not comparable

Granite 4.1 8b: —, o3: 40.7 (#35)

Multimodal benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Vision	—	1214
GeoBench	—	74%
VPCT	—	52%

Multilingual o3 leads

Granite 4.1 8b: 41.8 (#199), o3: 51.9 (#99)

Multilingual benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Non-English	1261	1402
LMArena Chinese	1335	1438
LMArena Russian	1238	1407
LMArena French	—	1430
LMArena German	—	1420
LMArena Japanese	—	1404
LMArena Korean	—	1371
LMArena Spanish	—	1395

Instruction Following o3 leads

Granite 4.1 8b: 67.0 (#198), o3: 73.0 (#122)

Instruction Following benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Instruction Following	1268	1368
IFEval	—	86.9%

Long Context o3 leads

Granite 4.1 8b: 38.7 (#188), o3: 53.2 (#6)

Long Context benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Longer Query	1275	1372
Fiction.LiveBench	—	88.9%
CL-bench	—	17.8%

Writing & Preference o3 leads

Granite 4.1 8b: 48.3 (#199), o3: 63.7 (#62)

Writing & Preference benchmarks
Benchmark	Granite 4.1 8b	o3
LMArena Text	1291	1410
LMArena Creative Writing	1249	1359
LMArena Multi-Turn	1269	1405
Short-Story Creative Writing	—	83.9%
EQ-Bench Creative Writing	—	1676
WildBench	—	86.1%

Frequently asked questions

Is Granite 4.1 8b better than o3?