Noometry

Model comparison

o1 vs Qwen2.5-Max

o1 and Qwen2.5-Max score almost the same on the Noometry Index (40.9 vs 40.7), so choose on price, context window or the category you care about most.

Last verified . 27 shared benchmarks.

o1 OpenAI

40.9

Rank #143 Confirmed

Qwen2.5-Max Alibaba (Qwen)

40.7

Rank #146 Confirmed

Summary

  • They share 27 benchmarks with published results for both. o1 scores higher in 7 categories and Qwen2.5-Max in 1 category; 5 gaps are clear of the uncertainty.
  • The widest gap is in long context, where o1 leads 50.3 to 41.4.
  • The biggest single-benchmark swing is LiveBench Reasoning: 91.6% for o1 and 51.4% for Qwen2.5-Max.

Side by side

o1Qwen2.5-Max
ProviderOpenAIAlibaba (Qwen)
Noometry Index40.940.7
Released2024-09-122025-01-25
WeightsProprietaryProprietary
Context window200K—
Max output100K—
Input $ / M tokens$15—
Output $ / M tokens$60—
Results tracked5227

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o1 leads

o1: 46.1 (#70), Qwen2.5-Max: 41.8 (#117)

Benchmarko1Qwen2.5-Max
LiveBench Coding69.7%64.4%
LMArena Coding13671359
Aider Polyglot61.7%—
WeirdML47.6%—
CadEval56%—
HumanEval+89%—
MBPP+80.2%—

Agentic & Tool Use Not comparable

o1: 24.6 (#117), Qwen2.5-Max: —

Benchmarko1Qwen2.5-Max
Cybench10%—
METR Time Horizons51.1%—

Reasoning o1 leads

o1: 27.9 (#111), Qwen2.5-Max: 25.6 (#147)

Benchmarko1Qwen2.5-Max
LiveBench Reasoning91.6%51.4%
LMArena Hard Prompts13711360
LiveBench Data Analysis65.5%67.9%
Epoch Capabilities Index141.91132.53
LiveBench75.7%62.3%
SimpleBench41.7%—
ARC-AGI-130.7%—
Chess Puzzles15%—
EnigmaEval5.7%—
DTBench74.7%—
LMCA22.3%—

Math Too close to call

o1: 36.1 (#175), Qwen2.5-Max: 36.9 (#162)

Benchmarko1Qwen2.5-Max
LiveBench Math80.3%58.4%
LMArena Math13881369
FrontierMath (Tiers 1-3)14.7%—
OTIS Mock AIME 2024-202573.3%—
MATH Level 594.7%—
FrontierMath (Feb 2025 set)9.3%—

Knowledge o1 leads

o1: 41.5 (#110), Qwen2.5-Max: 35.3 (#186)

Benchmarko1Qwen2.5-Max
Confabulations11.7%21.8%
LMArena Expert13611337
GPQA Diamond76.8%—
Humanity's Last Exam8%—
SimpleQA Verified41.1%—

Multimodal Not comparable

o1: 34.2 (#93), Qwen2.5-Max: —

Benchmarko1Qwen2.5-Max
LMArena Vision1168—
GeoBench80%—
VPCT37%—
SpatialViz-Bench41.4%—

Multilingual Too close to call

o1: 48.6 (#142), Qwen2.5-Max: 48.1 (#146)

Benchmarko1Qwen2.5-Max
LMArena Non-English13581352
LMArena Chinese13941382
LMArena French13441396
LMArena German13371350
LMArena Japanese13461300
LMArena Korean13961304
LMArena Russian13561353
LMArena Spanish13451377

Instruction Following o1 leads

o1: 74.8 (#86), Qwen2.5-Max: 71.3 (#152)

Benchmarko1Qwen2.5-Max
LiveBench Instruction Following81.5%75.3%
LMArena Instruction Following13671335

Long Context o1 leads

o1: 50.3 (#9), Qwen2.5-Max: 41.4 (#142)

Benchmarko1Qwen2.5-Max
LMArena Longer Query13781358
Fiction.LiveBench83.3%—

Writing & Preference Too close to call

o1: 55.6 (#144), Qwen2.5-Max: 55.4 (#146)

Benchmarko1Qwen2.5-Max
LMArena Text13661367
LMArena Creative Writing13481339
Short-Story Creative Writing70.2%72.9%
LMArena Multi-Turn13691364
LiveBench Language65.4%56.3%

Frequently asked questions

Is o1 better than Qwen2.5-Max?

o1 and Qwen2.5-Max score almost the same on the Noometry Index (40.9 vs 40.7), so choose on price, context window or the category you care about most.

Is o1 or Qwen2.5-Max better for coding?

o1 scores higher on coding benchmarks: 46.1 versus 41.8 in the Noometry coding category.

How many benchmarks do o1 and Qwen2.5-Max share?

27 benchmarks have published results for both models. o1 has 52 scored results on Noometry and Qwen2.5-Max has 27.

Related comparisons

Go deeper