Noometry

Model comparison

GPT-4 vs Qwen Max

Qwen Max is the stronger model overall, scoring 34.4 to 28.9 on the Noometry Index.

Last verified . 20 shared benchmarks.

GPT-4 OpenAI

28.9

Rank #308 Confirmed

Qwen Max Alibaba (Qwen)

34.4

Rank #226 Confirmed

Summary

  • They share 20 benchmarks with published results for both. GPT-4 scores higher in 1 category and Qwen Max in 7 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Qwen Max leads 48.0 to 35.0.
  • The biggest single-benchmark swing is MATH Level 5: 23% for GPT-4 and 67.2% for Qwen Max.
  • Qwen Max is cheaper at $1.60 / $6.40 per million input/output tokens, against $30 / $60 for GPT-4.
  • Qwen Max accepts more context: 33K tokens versus 8K.

Side by side

GPT-4Qwen Max
ProviderOpenAIAlibaba (Qwen)
Noometry Index28.934.4
Released2023-03-142024-04-03
WeightsProprietaryProprietary
Context window8K33K
Max output8K8K
Input $ / M tokens$30$1.60
Output $ / M tokens$60$6.40
Results tracked3823

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

GPT-4: 31.2 (#277), Qwen Max: 30.3 (#288)

BenchmarkGPT-4Qwen Max
LMArena Coding12541288
Aider Polyglot—21.8%
WeirdML12.4%—
BigCodeBench Instruct46%—
BigCodeBench Complete57.2%—
HumanEval+79.3%—

Agentic & Tool Use Not comparable

GPT-4: —, Qwen Max: —

BenchmarkGPT-4Qwen Max
METR Time Horizons36.1%—

Reasoning Qwen Max leads

GPT-4: 17.3 (#285), Qwen Max: 24.1 (#154)

BenchmarkGPT-4Qwen Max
LMArena Hard Prompts12411270
Chess Puzzles4%—
Mystery Game Puzzles12%—
DTBench62.7%—
LMCA17.1%—
BIG-Bench Hard75.1%—
Epoch Capabilities Index125.88—
ForecastBench57.8—
HellaSwag95.3%—
WinoGrande87.5%—

Math Qwen Max leads

GPT-4: 10.1 (#304), Qwen Max: 21.0 (#272)

BenchmarkGPT-4Qwen Max
OTIS Mock AIME 2024-20251.1%16.1%
LMArena Math12691275
MATH Level 523%67.2%
FrontierMath (Feb 2025 set)—1%
GSM8K92%—

Knowledge Qwen Max leads

GPT-4: 18.4 (#277), Qwen Max: 30.3 (#222)

BenchmarkGPT-4Qwen Max
GPQA Diamond35.7%56.1%
LMArena Expert12111248
MMLU86.4%—
TriviaQA84.8%—

Multilingual Qwen Max leads

GPT-4: 40.8 (#210), Qwen Max: 42.0 (#197)

BenchmarkGPT-4Qwen Max
LMArena Non-English12461263
LMArena Chinese12421254
LMArena French12821329
LMArena German12511254
LMArena Japanese12091205
LMArena Korean11841141
LMArena Russian12511274
LMArena Spanish12611290

Instruction Following Qwen Max leads

GPT-4: 65.5 (#217), Qwen Max: 66.7 (#203)

BenchmarkGPT-4Qwen Max
LMArena Instruction Following12411262

Long Context Qwen Max leads

GPT-4: 37.7 (#207), Qwen Max: 39.4 (#175)

BenchmarkGPT-4Qwen Max
LMArena Longer Query12441289
Fiction.LiveBench—66.7%

Writing & Preference Qwen Max leads

GPT-4: 35.0 (#263), Qwen Max: 48.0 (#200)

BenchmarkGPT-4Qwen Max
LMArena Text12631282
LMArena Creative Writing12441248
LMArena Multi-Turn12571277
EQ-Bench Creative Writing752—

Frequently asked questions

Is GPT-4 better than Qwen Max?

Qwen Max is the stronger model overall, scoring 34.4 to 28.9 on the Noometry Index.

Which is cheaper, GPT-4 or Qwen Max?

Qwen Max is cheaper. It lists at $1.60 per million input tokens and $6.40 per million output tokens; GPT-4 lists at $30 and $60.

Is GPT-4 or Qwen Max better for coding?

They score almost the same on coding (31.2 vs 30.3); test both on your own repository before choosing.

Which has the bigger context window?

Qwen Max does, with 33K tokens against 8K.

How many benchmarks do GPT-4 and Qwen Max share?

20 benchmarks have published results for both models. GPT-4 has 38 scored results on Noometry and Qwen Max has 23.

Related comparisons

Go deeper