Model comparison

ERNIE 5.1 vs o3

o3 is the stronger model overall, scoring 47.5 to 43.8 on the Noometry Index.

Last verified October 10, 2026. 18 shared benchmarks.

ERNIE 5.1

Baidu

43.8

Rank #83 Confirmed

o3 OpenAI

47.5

Rank #61 Confirmed

Summary

They share 18 benchmarks with published results for both. ERNIE 5.1 scores higher in 3 categories and o3 in 5 categories; 8 gaps are clear of the uncertainty.
The widest gap is in knowledge, where o3 leads 54.6 to 41.9.

Side by side

ERNIE 5.1 and o3 specifications
	ERNIE 5.1	o3
Provider	Baidu	OpenAI
Noometry Index	43.8	47.5
Released	—	2025-04-16
Weights	Proprietary	Proprietary
Context window	—	200K
Max output	—	100K
Input $ / M tokens	—	$2
Output $ / M tokens	—	$8
Results tracked	19	63

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o3 leads

ERNIE 5.1: 44.0 (#76), o3: 46.8 (#64)

Coding benchmarks
Benchmark	ERNIE 5.1	o3
LMArena Coding	1488	1408
SWE-bench Verified	—	62.3%
SWE-bench Verified (bash only)	—	58.4%
Aider Polyglot	—	81.3%
GSO	—	8.8%
WeirdML	—	52.4%
CadEval	—	74%
ALE-Bench	—	933.55

Agentic & Tool Use Not comparable

ERNIE 5.1: —, o3: 34.5 (#44)

Agentic & Tool Use benchmarks
Benchmark	ERNIE 5.1	o3
LMArena Search	1227	1144
Berkeley Function Calling Leaderboard	—	63%
GDPval	—	30.8%
DeepResearch Bench	—	45.2%
OSWorld	—	23%
METR Time Horizons	—	65.4%

Reasoning o3 leads

ERNIE 5.1: 21.9 (#211), o3: 32.0 (#78)

Reasoning benchmarks
Benchmark	ERNIE 5.1	o3
LMArena Hard Prompts	1481	1402
ARC-AGI-2	—	6.5%
SimpleBench	—	53.1%
Kagi LLM Benchmark	—	67.6%
NYT Connections (extended)	23.4%	—
ARC-AGI-1	—	60.8%
CritPt	—	1.4%
Chess Puzzles	—	38%
EnigmaEval	—	13.1%
Mystery Game Puzzles	—	29%
DTBench	—	84.8%
LMCA	—	39.7%
Epoch Capabilities Index	—	146.86
ForecastBench	—	62.5

Math o3 leads

ERNIE 5.1: 40.3 (#92), o3: 50.2 (#58)

Math benchmarks
Benchmark	ERNIE 5.1	o3
LMArena Math	1481	1426
FrontierMath (Tiers 1-3)	—	33.3%
OTIS Mock AIME 2024-2025	—	84.4%
Omni-MATH	—	71.4%
MATH Level 5	—	97.8%
FrontierMath (Feb 2025 set)	—	18.7%
FrontierMath Tier 4 (v1)	—	2.1%

Knowledge o3 leads

ERNIE 5.1: 41.9 (#102), o3: 54.6 (#52)

Knowledge benchmarks
Benchmark	ERNIE 5.1	o3
LMArena Expert	1492	1402
GPQA Diamond	—	81.8%
Humanity's Last Exam	—	20.3%
SimpleQA Verified	—	49.4%
MMLU-Pro	—	85.9%
Confabulations	—	14.4%
GPQA (HELM)	—	75.3%

Multimodal Not comparable

ERNIE 5.1: —, o3: 41.4 (#36)

Multimodal benchmarks
Benchmark	ERNIE 5.1	o3
LMArena Vision	—	1214
GeoBench	—	74%
VPCT	—	52%

Multilingual ERNIE 5.1 leads

ERNIE 5.1: 55.5 (#29), o3: 51.7 (#105)

Multilingual benchmarks
Benchmark	ERNIE 5.1	o3
LMArena Non-English	1454	1401
LMArena Chinese	1508	1437
LMArena French	1488	1430
LMArena German	1470	1420
LMArena Japanese	1422	1403
LMArena Korean	1427	1370
LMArena Russian	1459	1406
LMArena Spanish	1473	1395

Instruction Following ERNIE 5.1 leads

ERNIE 5.1: 76.7 (#37), o3: 72.8 (#127)

Instruction Following benchmarks
Benchmark	ERNIE 5.1	o3
LMArena Instruction Following	1460	1368
IFEval	—	86.9%

Long Context o3 leads

ERNIE 5.1: 44.7 (#59), o3: 53.3 (#6)

Long Context benchmarks
Benchmark	ERNIE 5.1	o3
LMArena Longer Query	1462	1372
Fiction.LiveBench	—	88.9%
CL-bench	—	17.8%

Writing & Preference ERNIE 5.1 leads

ERNIE 5.1: 65.1 (#52), o3: 63.5 (#64)

Writing & Preference benchmarks
Benchmark	ERNIE 5.1	o3
LMArena Text	1468	1410
LMArena Creative Writing	1441	1359
LMArena Multi-Turn	1471	1405
Short-Story Creative Writing	—	83.9%
EQ-Bench Creative Writing	—	1676
WildBench	—	86.1%

Frequently asked questions

Is ERNIE 5.1 better than o3?