Noometry

Model comparison

Pixtral Large vs Qwen1.5-72B

Pixtral Large is the stronger model overall, scoring 32.2 to 30.8 on the Noometry Index.

Last verified . 0 shared benchmarks.

Pixtral Large Mistral AI

32.2

Rank #259 Reported

Qwen1.5-72B Alibaba (Qwen)

30.8

Rank #285 Confirmed

Summary

  • The widest gap is in writing & preference, where Qwen1.5-72B leads 37.3 to 32.9.

Side by side

Pixtral LargeQwen1.5-72B
ProviderMistral AIAlibaba (Qwen)
Noometry Index32.230.8
Released2024-11-012024-02-04
WeightsOpenOpen
Context window128K—
Max output128K—
Input $ / M tokens$2—
Output $ / M tokens$6—
Results tracked322

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Pixtral Large: —, Qwen1.5-72B: 31.9 (#277)

BenchmarkPixtral LargeQwen1.5-72B
BigCodeBench Instruct—33.2%
LMArena Coding—1165
BigCodeBench Complete—40.3%
HumanEval+—59.1%
MBPP+—61.6%

Reasoning Too close to call

Pixtral Large: 21.7 (#218), Qwen1.5-72B: 22.2 (#203)

BenchmarkPixtral LargeQwen1.5-72B
EnigmaEval0.8%—
LMArena Hard Prompts—1148

Math Not comparable

Pixtral Large: —, Qwen1.5-72B: 33.2 (#205)

BenchmarkPixtral LargeQwen1.5-72B
LMArena Math—1164

Knowledge Not comparable

Pixtral Large: —, Qwen1.5-72B: 11.5 (#300)

BenchmarkPixtral LargeQwen1.5-72B
GPQA Diamond—28.8%
LMArena Expert—1136

Multimodal Not comparable

Pixtral Large: 30.6 (#111), Qwen1.5-72B: —

BenchmarkPixtral LargeQwen1.5-72B
LMArena Vision1089—

Multilingual Not comparable

Pixtral Large: —, Qwen1.5-72B: 33.2 (#253)

BenchmarkPixtral LargeQwen1.5-72B
LMArena Non-English—1135
LMArena Chinese—1186
LMArena French—1159
LMArena German—1084
LMArena Japanese—1061
LMArena Korean—1050
LMArena Russian—1104
LMArena Spanish—1110

Instruction Following Not comparable

Pixtral Large: —, Qwen1.5-72B: 59.3 (#256)

BenchmarkPixtral LargeQwen1.5-72B
LMArena Instruction Following—1141

Long Context Not comparable

Pixtral Large: —, Qwen1.5-72B: 35.1 (#243)

BenchmarkPixtral LargeQwen1.5-72B
LMArena Longer Query—1157

Writing & Preference Qwen1.5-72B leads

Pixtral Large: 32.9 (#278), Qwen1.5-72B: 37.3 (#258)

BenchmarkPixtral LargeQwen1.5-72B
LMArena Text—1166
LMArena Creative Writing—1137
EQ-Bench Creative Writing988—
LMArena Multi-Turn—1160

Frequently asked questions

Is Pixtral Large better than Qwen1.5-72B?

Pixtral Large is the stronger model overall, scoring 32.2 to 30.8 on the Noometry Index.

How many benchmarks do Pixtral Large and Qwen1.5-72B share?

0 benchmarks have published results for both models. Pixtral Large has 3 scored results on Noometry and Qwen1.5-72B has 22.

Related comparisons

Go deeper