Noometry

Model comparison

Gemma 1.1 2b IT vs Pixtral Large

Pixtral Large is the stronger model overall, scoring 31.7 to 28.8 on the Noometry Index.

Last verified . 0 shared benchmarks.

Gemma 1.1 2b IT Google

28.8

Rank #313 Confirmed

Pixtral Large Mistral AI

31.7

Rank #257 Reported

Summary

  • The widest gap is in writing & preference, where Pixtral Large leads 32.9 to 25.3.

Side by side

Gemma 1.1 2b ITPixtral Large
ProviderGoogleMistral AI
Noometry Index28.831.7
Released—2024-11-01
WeightsOpenOpen
Context window—128K
Max output—128K
Input $ / M tokens—$2
Output $ / M tokens—$6
Results tracked163

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Gemma 1.1 2b IT: 29.6 (#294), Pixtral Large: —

BenchmarkGemma 1.1 2b ITPixtral Large
LMArena Coding1034—
HumanEval+17.7%—
MBPP+23.3%—

Reasoning Pixtral Large leads

Gemma 1.1 2b IT: 18.2 (#268), Pixtral Large: 20.9 (#208)

BenchmarkGemma 1.1 2b ITPixtral Large
EnigmaEval—0.8%
LMArena Hard Prompts1005—

Math Not comparable

Gemma 1.1 2b IT: 28.3 (#230), Pixtral Large: —

BenchmarkGemma 1.1 2b ITPixtral Large
LMArena Math1047—

Knowledge Not comparable

Gemma 1.1 2b IT: 26.0 (#255), Pixtral Large: —

BenchmarkGemma 1.1 2b ITPixtral Large
LMArena Expert970—

Multimodal Not comparable

Gemma 1.1 2b IT: —, Pixtral Large: 30.0 (#105)

BenchmarkGemma 1.1 2b ITPixtral Large
LMArena Vision—1089

Multilingual Not comparable

Gemma 1.1 2b IT: 24.7 (#284), Pixtral Large: —

BenchmarkGemma 1.1 2b ITPixtral Large
LMArena Non-English988—
LMArena Chinese1011—
LMArena German944—
LMArena Korean899—
LMArena Russian990—

Instruction Following Not comparable

Gemma 1.1 2b IT: 50.1 (#294), Pixtral Large: —

BenchmarkGemma 1.1 2b ITPixtral Large
LMArena Instruction Following992—

Long Context Not comparable

Gemma 1.1 2b IT: 30.5 (#281), Pixtral Large: —

BenchmarkGemma 1.1 2b ITPixtral Large
LMArena Longer Query1003—

Writing & Preference Pixtral Large leads

Gemma 1.1 2b IT: 25.3 (#301), Pixtral Large: 32.9 (#273)

BenchmarkGemma 1.1 2b ITPixtral Large
LMArena Text1022—
LMArena Creative Writing998—
EQ-Bench Creative Writing—988
LMArena Multi-Turn958—

Frequently asked questions

Is Gemma 1.1 2b IT better than Pixtral Large?

Pixtral Large is the stronger model overall, scoring 31.7 to 28.8 on the Noometry Index.

How many benchmarks do Gemma 1.1 2b IT and Pixtral Large share?

0 benchmarks have published results for both models. Gemma 1.1 2b IT has 16 scored results on Noometry and Pixtral Large has 3.

Related comparisons

Go deeper