Model comparison

o1 vs Qwen3-4B

o1 is the stronger model overall, scoring 40.9 to 31.9 on the Noometry Index.

Last verified October 11, 2026. 3 shared benchmarks.

o1 OpenAI

40.9

Rank #143 Confirmed

Qwen3-4B

Alibaba (Qwen)

31.9

Rank #264 Confirmed

Summary

They share 3 benchmarks with published results for both. o1 scores higher in 3 categories and Qwen3-4B in 1 category; 4 gaps are clear of the uncertainty.
The widest gap is in reasoning, where o1 leads 27.9 to 19.2.
The biggest single-benchmark swing is GPQA Diamond: 76.8% for o1 and 52.3% for Qwen3-4B.
Qwen3-4B has downloadable open weights; the other is API-only.

Side by side

o1 and Qwen3-4B specifications
	o1	Qwen3-4B
Provider	OpenAI	Alibaba (Qwen)
Noometry Index	40.9	31.9
Released	2024-09-12	2025-04-29
Weights	Proprietary	Open
Context window	200K	—
Max output	100K	—
Input $ / M tokens	$15	—
Output $ / M tokens	$60	—
Results tracked	52	6

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

o1: 46.1 (#70), Qwen3-4B: —

Coding benchmarks
Benchmark	o1	Qwen3-4B
Aider Polyglot	61.7%	—
WeirdML	47.6%	—
LiveBench Coding	69.7%	—
LMArena Coding	1367	—
CadEval	56%	—
HumanEval+	89%	—
MBPP+	80.2%	—

Agentic & Tool Use Qwen3-4B leads

o1: 24.6 (#117), Qwen3-4B: 27.6 (#100)

Agentic & Tool Use benchmarks
Benchmark	o1	Qwen3-4B
Berkeley Function Calling Leaderboard	—	35.7%
Cybench	10%	—
METR Time Horizons	51.1%	—

Reasoning o1 leads

o1: 27.9 (#111), Qwen3-4B: 19.2 (#268)

Reasoning benchmarks
Benchmark	o1	Qwen3-4B
Chess Puzzles	15%	4%
SimpleBench	41.7%	—
ARC-AGI-1	30.7%	—
EnigmaEval	5.7%	—
LiveBench Reasoning	91.6%	—
LMArena Hard Prompts	1371	—
DTBench	74.7%	—
LiveBench Data Analysis	65.5%	—
LMCA	22.3%	—
Epoch Capabilities Index	141.91	—
LiveBench	75.7%	—

Math o1 leads

o1: 36.1 (#175), Qwen3-4B: 29.7 (#240)

Math benchmarks
Benchmark	o1	Qwen3-4B
OTIS Mock AIME 2024-2025	73.3%	52.2%
FrontierMath (Tiers 1-3)	14.7%	—
MathArena Final-Answer Competitions	—	38.5%
LiveBench Math	80.3%	—
LMArena Math	1388	—
MATH Level 5	94.7%	—
FrontierMath (Feb 2025 set)	9.3%	—

Knowledge o1 leads

o1: 41.5 (#110), Qwen3-4B: 33.0 (#208)

Knowledge benchmarks
Benchmark	o1	Qwen3-4B
GPQA Diamond	76.8%	52.3%
Humanity's Last Exam	8%	—
SimpleQA Verified	41.1%	—
Confabulations	11.7%	—
Vectara Hallucination Rate	—	5.7%
LMArena Expert	1361	—

Multimodal Not comparable

o1: 34.2 (#93), Qwen3-4B: —

Multimodal benchmarks
Benchmark	o1	Qwen3-4B
LMArena Vision	1168	—
GeoBench	80%	—
VPCT	37%	—
SpatialViz-Bench	41.4%	—

Multilingual Not comparable

o1: 48.6 (#142), Qwen3-4B: —

Multilingual benchmarks
Benchmark	o1	Qwen3-4B
LMArena Non-English	1358	—
LMArena Chinese	1394	—
LMArena French	1344	—
LMArena German	1337	—
LMArena Japanese	1346	—
LMArena Korean	1396	—
LMArena Russian	1356	—
LMArena Spanish	1345	—

Instruction Following Not comparable

o1: 74.8 (#86), Qwen3-4B: —

Instruction Following benchmarks
Benchmark	o1	Qwen3-4B
LiveBench Instruction Following	81.5%	—
LMArena Instruction Following	1367	—

Long Context Not comparable

o1: 50.3 (#9), Qwen3-4B: —

Long Context benchmarks
Benchmark	o1	Qwen3-4B
Fiction.LiveBench	83.3%	—
LMArena Longer Query	1378	—

Writing & Preference Not comparable

o1: 55.6 (#144), Qwen3-4B: —

Writing & Preference benchmarks
Benchmark	o1	Qwen3-4B
LMArena Text	1366	—
LMArena Creative Writing	1348	—
Short-Story Creative Writing	70.2%	—
LMArena Multi-Turn	1369	—
LiveBench Language	65.4%	—

Frequently asked questions

Is o1 better than Qwen3-4B?