Noometry

Model comparison

o1 vs QwQ-32B

o1 is the stronger model overall, scoring 40.9 to 39.8 on the Noometry Index.

Last verified . 32 shared benchmarks.

o1 OpenAI

40.9

Rank #143 Confirmed

QwQ-32B Alibaba (Qwen)

39.8

Rank #159 Confirmed

Summary

  • They share 32 benchmarks with published results for both. o1 scores higher in 7 categories and QwQ-32B in 1 category; 8 gaps are clear of the uncertainty.
  • The widest gap is in coding, where o1 leads 46.1 to 35.4.
  • The biggest single-benchmark swing is Aider Polyglot: 61.7% for o1 and 20.9% for QwQ-32B.
  • QwQ-32B has downloadable open weights; the other is API-only.

Side by side

o1QwQ-32B
ProviderOpenAIAlibaba (Qwen)
Noometry Index40.939.8
Released2024-09-122024-11-28
WeightsProprietaryOpen
Context window200K—
Max output100K—
Input $ / M tokens$15—
Output $ / M tokens$60—
Results tracked5236

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o1 leads

o1: 46.1 (#70), QwQ-32B: 35.4 (#226)

Benchmarko1QwQ-32B
Aider Polyglot61.7%20.9%
LiveBench Coding69.7%72.2%
LMArena Coding13671333
WeirdML47.6%—
BigCodeBench Instruct—44.6%
BigCodeBench Complete—54.4%
CadEval56%—
HumanEval+89%—
MBPP+80.2%—

Agentic & Tool Use Not comparable

o1: 24.6 (#117), QwQ-32B: —

Benchmarko1QwQ-32B
Cybench10%—
METR Time Horizons51.1%—

Reasoning o1 leads

o1: 27.9 (#111), QwQ-32B: 23.7 (#174)

Benchmarko1QwQ-32B
Chess Puzzles15%5%
LiveBench Reasoning91.6%83.5%
LMArena Hard Prompts13711325
LiveBench Data Analysis65.5%65%
Epoch Capabilities Index141.91137.6
LiveBench75.7%72%
SimpleBench41.7%—
ARC-AGI-130.7%—
EnigmaEval5.7%—
DTBench74.7%—
LMCA22.3%—
ForecastBench—58.3

Math QwQ-32B leads

o1: 36.1 (#175), QwQ-32B: 38.0 (#143)

Benchmarko1QwQ-32B
OTIS Mock AIME 2024-202573.3%59.2%
LiveBench Math80.3%77.8%
LMArena Math13881359
FrontierMath (Tiers 1-3)14.7%—
MATH Level 594.7%—
FrontierMath (Feb 2025 set)9.3%—

Knowledge o1 leads

o1: 41.5 (#110), QwQ-32B: 37.2 (#158)

Benchmarko1QwQ-32B
GPQA Diamond76.8%65.3%
Confabulations11.7%15.6%
LMArena Expert13611324
Humanity's Last Exam8%—
SimpleQA Verified41.1%—

Multimodal Not comparable

o1: 34.2 (#93), QwQ-32B: —

Benchmarko1QwQ-32B
LMArena Vision1168—
GeoBench80%—
VPCT37%—
SpatialViz-Bench41.4%—

Multilingual o1 leads

o1: 48.6 (#142), QwQ-32B: 44.8 (#176)

Benchmarko1QwQ-32B
LMArena Non-English13581305
LMArena Chinese13941378
LMArena French13441336
LMArena German13371313
LMArena Japanese13461262
LMArena Korean13961279
LMArena Russian13561297
LMArena Spanish13451354

Instruction Following o1 leads

o1: 74.8 (#86), QwQ-32B: 72.6 (#137)

Benchmarko1QwQ-32B
LiveBench Instruction Following81.5%81.8%
LMArena Instruction Following13671297

Long Context o1 leads

o1: 50.3 (#9), QwQ-32B: 49.0 (#11)

Benchmarko1QwQ-32B
Fiction.LiveBench83.3%83.3%
LMArena Longer Query13781308

Writing & Preference o1 leads

o1: 55.6 (#144), QwQ-32B: 50.6 (#180)

Benchmarko1QwQ-32B
LMArena Text13661329
LMArena Creative Writing13481288
Short-Story Creative Writing70.2%80.2%
LMArena Multi-Turn13691314
LiveBench Language65.4%51.4%
EQ-Bench Creative Writing—1257

Frequently asked questions

Is o1 better than QwQ-32B?

o1 is the stronger model overall, scoring 40.9 to 39.8 on the Noometry Index.

Is o1 or QwQ-32B better for coding?

o1 scores higher on coding benchmarks: 46.1 versus 35.4 in the Noometry coding category.

How many benchmarks do o1 and QwQ-32B share?

32 benchmarks have published results for both models. o1 has 52 scored results on Noometry and QwQ-32B has 36.

Related comparisons

Go deeper