Noometry

Model comparison

o1-mini vs Qwen1.5-110B

o1-mini and Qwen1.5-110B score almost the same on the Noometry Index (34.0 vs 34.2), so choose on price, context window or the category you care about most.

Last verified . 17 shared benchmarks.

o1-mini OpenAI

34.0

Rank #235 Confirmed

Qwen1.5-110B Alibaba (Qwen)

34.2

Rank #234 Confirmed

Summary

  • They share 17 benchmarks with published results for both. o1-mini scores higher in 7 categories and Qwen1.5-110B in 1 category; 8 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Qwen1.5-110B leads 22.7 to 8.8.
  • Qwen1.5-110B has downloadable open weights; the other is API-only.

Side by side

o1-miniQwen1.5-110B
ProviderOpenAIAlibaba (Qwen)
Noometry Index34.034.2
Released2024-09-122024-04-25
WeightsProprietaryOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked3920

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding o1-mini leads

o1-mini: 35.5 (#224), Qwen1.5-110B: 33.0 (#264)

Benchmarko1-miniQwen1.5-110B
LMArena Coding13621184
Aider Polyglot32.9%—
WeirdML36.3%—
BigCodeBench Instruct—35%
LiveBench Coding48%—
BigCodeBench Complete—44.4%
HumanEval+89%—
MBPP+78.8%—

Agentic & Tool Use Not comparable

o1-mini: 24.6 (#118), Qwen1.5-110B: —

Benchmarko1-miniQwen1.5-110B
Cybench10%—

Reasoning Qwen1.5-110B leads

o1-mini: 8.8 (#346), Qwen1.5-110B: 22.7 (#189)

Benchmarko1-miniQwen1.5-110B
LMArena Hard Prompts13331168
ARC-AGI-20.8%—
SimpleBench18.1%—
ARC-AGI-114%—
LiveBench Reasoning72.3%—
LiveBench Data Analysis57.9%—
Epoch Capabilities Index135.82—
ForecastBench—57.7
LiveBench57.8%—

Math o1-mini leads

o1-mini: 35.4 (#186), Qwen1.5-110B: 33.7 (#201)

Benchmarko1-miniQwen1.5-110B
LMArena Math13581185
OTIS Mock AIME 2024-202546.9%—
LiveBench Math62%—
MATH Level 589.2%—
FrontierMath (Feb 2025 set)1.7%—

Knowledge o1-mini leads

o1-mini: 34.9 (#192), Qwen1.5-110B: 31.2 (#219)

Benchmarko1-miniQwen1.5-110B
LMArena Expert13161144
GPQA Diamond62.4%—
Confabulations18.6%—

Multilingual o1-mini leads

o1-mini: 43.6 (#182), Qwen1.5-110B: 33.6 (#250)

Benchmarko1-miniQwen1.5-110B
LMArena Non-English12891142
LMArena Chinese13141206
LMArena French12931151
LMArena German12781123
LMArena Japanese12451074
LMArena Korean12231044
LMArena Russian12831118
LMArena Spanish13031142

Instruction Following o1-mini leads

o1-mini: 66.7 (#206), Qwen1.5-110B: 60.3 (#252)

Benchmarko1-miniQwen1.5-110B
LMArena Instruction Following13041158
LiveBench Instruction Following65.4%—

Long Context o1-mini leads

o1-mini: 40.1 (#161), Qwen1.5-110B: 35.1 (#242)

Benchmarko1-miniQwen1.5-110B
LMArena Longer Query13201157

Writing & Preference o1-mini leads

o1-mini: 48.4 (#202), Qwen1.5-110B: 38.0 (#255)

Benchmarko1-miniQwen1.5-110B
LMArena Text13171175
LMArena Creative Writing12441148
LMArena Multi-Turn13141160
Short-Story Creative Writing64.9%—
LiveBench Language40.9%—

Frequently asked questions

Is o1-mini better than Qwen1.5-110B?

o1-mini and Qwen1.5-110B score almost the same on the Noometry Index (34.0 vs 34.2), so choose on price, context window or the category you care about most.

Is o1-mini or Qwen1.5-110B better for coding?

o1-mini scores higher on coding benchmarks: 35.5 versus 33.0 in the Noometry coding category.

How many benchmarks do o1-mini and Qwen1.5-110B share?

17 benchmarks have published results for both models. o1-mini has 39 scored results on Noometry and Qwen1.5-110B has 20.

Related comparisons

Go deeper