Model comparison

Gemini 1.0 Pro vs Mercury 2

Mercury 2 is the stronger model overall, scoring 38.8 to 27.1 on the Noometry Index.

Last verified . 11 shared benchmarks.

Gemini 1.0 Pro Google

27.1

Rank #327 Confirmed

Mercury 2 Inception

38.8

Rank #167 Confirmed

Summary

  • They share 11 benchmarks with published results for both. Gemini 1.0 Pro scores higher in 0 categories and Mercury 2 in 7 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Mercury 2 leads 35.6 to 15.5.

Side by side

Gemini 1.0 Pro and Mercury 2 specifications
Gemini 1.0 ProMercury 2
ProviderGoogleInception
Noometry Index27.138.8
Released2023-12-132026-02-20
WeightsProprietaryProprietary
Context window—128K
Max output—50K
Input $ / M tokens—$0.25
Output $ / M tokens—$0.75
Results tracked2417

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mercury 2 leads

Gemini 1.0 Pro: 31.6 (#270), Mercury 2: 33.2 (#250)

Coding benchmarks
BenchmarkGemini 1.0 ProMercury 2
LMArena Coding11081391
LMArena WebDev—1170
SciCode—38.7%
WeirdML—43.2%
ALE-Bench—785.58
HumanEval+55.5%—
MBPP+61.4%—

Reasoning Mercury 2 leads

Gemini 1.0 Pro: 16.4 (#292), Mercury 2: 23.0 (#170)

Reasoning benchmarks
BenchmarkGemini 1.0 ProMercury 2
LMArena Hard Prompts11091362
CritPt—0.8%
DTBench45.9%—
Epoch Capabilities Index116.97—

Math Not comparable

Gemini 1.0 Pro: 8.7 (#315), Mercury 2: —

Math benchmarks
BenchmarkGemini 1.0 ProMercury 2
OTIS Mock AIME 2024-20251.1%—
LMArena Math1132—
MATH Level 511.2%—

Knowledge Mercury 2 leads

Gemini 1.0 Pro: 15.5 (#286), Mercury 2: 35.6 (#170)

Knowledge benchmarks
BenchmarkGemini 1.0 ProMercury 2
LMArena Expert10591358
GPQA Diamond34%—
Vectara Hallucination Rate—12.3%
MMLU70%—

Multilingual Mercury 2 leads

Gemini 1.0 Pro: 33.5 (#247), Mercury 2: 46.8 (#153)

Multilingual benchmarks
BenchmarkGemini 1.0 ProMercury 2
LMArena Non-English11381331
LMArena Chinese11241416
LMArena Russian11861303
LMArena French1144—
LMArena German1125—
LMArena Japanese1023—
LMArena Spanish1118—

Instruction Following Mercury 2 leads

Gemini 1.0 Pro: 57.9 (#262), Mercury 2: 70.4 (#161)

Instruction Following benchmarks
BenchmarkGemini 1.0 ProMercury 2
LMArena Instruction Following11141328

Long Context Mercury 2 leads

Gemini 1.0 Pro: 34.3 (#244), Mercury 2: 40.4 (#149)

Long Context benchmarks
BenchmarkGemini 1.0 ProMercury 2
LMArena Longer Query11321329

Writing & Preference Mercury 2 leads

Gemini 1.0 Pro: 36.2 (#259), Mercury 2: 54.0 (#151)

Writing & Preference benchmarks
BenchmarkGemini 1.0 ProMercury 2
LMArena Text11491355
LMArena Creative Writing11301289
LMArena Multi-Turn11391358

Frequently asked questions

Is Gemini 1.0 Pro better than Mercury 2?

Mercury 2 is the stronger model overall, scoring 38.8 to 27.1 on the Noometry Index.

Is Gemini 1.0 Pro or Mercury 2 better for coding?

Mercury 2 scores higher on coding benchmarks: 33.2 versus 31.6 in the Noometry coding category.

How many benchmarks do Gemini 1.0 Pro and Mercury 2 share?

11 benchmarks have published results for both models. Gemini 1.0 Pro has 24 scored results on Noometry and Mercury 2 has 17.

Related comparisons

Go deeper