Noometry

Google, proprietary

Gemini 1.5 Pro (May 2024)

Gemini 1.5 Pro (May 2024) by Google ranks 261st of 354 ranked models on the Noometry Index as of October 2026, with a score of 32.1. Its strongest category is multimodal, where it ranks 77th.

Last verified

Noometry rank
#261 of 354
Index score
32.1
Evidence
Confirmed 45 results
Provider
Google
Released
February 15, 2024
Weights
Proprietary
Reasoning
Unknown
Context window
—
Max output
—
Input price
Not listed
Output price
Not listed
Blended price
Not listed
Output speed
Not measured
Value
Not ranked
Knowledge cutoff
Unknown

Category scores

Each category score combines every public result we have in that category.

  1. Coding 34.2
  2. Agentic & Tool Use 17.9
  3. Reasoning 12.3
  4. Math 25.8
  5. Knowledge 29.4
  6. Multimodal 36.8
  7. Multilingual 45.3
  8. Instruction Following 68.6
  9. Long Context 39.8
  10. Writing & Preference 52.4
CategoryScoreRankResults
Coding34.2#2415
Agentic & Tool Use17.9#1453
Reasoning12.3#3384
Math25.8#2664
Knowledge29.4#2396
Multimodal36.8#772
Multilingual45.3#1741
Instruction Following68.6#1852
Long Context39.8#1691
Writing & Preference52.4#1724

Strengths and weaknesses

Categories where Gemini 1.5 Pro (May 2024) places highest and lowest among the models ranked in each, with its score against that category's median.

Strongest categories

CategoryScorevs medianRank
Writing & Preference52.4−1.4#172 of 312, top 56%
Long Context39.8−1.2#169 of 296, top 58%
Multilingual45.3−2.1#174 of 297, top 59%

Weakest categories

CategoryScorevs medianRank
Reasoning12.3−11.4#338 of 350, top 97%
Agentic & Tool Use17.9−12.4#145 of 154, top 95%
Math25.8−10.7#266 of 327, top 82%

Closest competitors

The models ranked just above and below Gemini 1.5 Pro (May 2024). When scores are this close, price and speed are often the better way to choose.

ModelRankScoreBlended $/MSpeed
Command R+#25732.4$4.38—Compare
Granite 3.1 8b Instruct#25832.4——Compare
Pixtral Large#25932.2$3—Compare
Falcon-180B#26032.2——Compare
Gemma 3 12B#26232.1$0.075—Compare
Mistral Large#26331.9$3—Compare
Qwen3-4B#26431.9——Compare
Amazon Nova Lite#26531.9$0.10—Compare

Sponsored placements are available on pages like this one. Advertise on Noometry

Benchmark results

Every published result we track, with its source. Bold rows are the ones used for ranking; where several exist we prefer independent runs over self-reported numbers.

Coding

BenchmarkScorePositionSettingSourceDate
WeirdML22.2%#103 of 119, top 87%Epoch AI
BigCodeBench Instruct43.8%#24 of 64, top 38%BigCodeBench2024-05-14
LMArena Coding1266LMArena2026-10-08
LMArena Coding1294#201 of 294, top 69%LMArena2026-10-08
BigCodeBench Complete57.5%#11 of 66, top 17%BigCodeBench2024-05-14
CadEval34%#9 of 14, top 65%Epoch AI
HumanEval+61%EvalPlus
HumanEval+79.3%#11 of 45, top 25%EvalPlus
MBPP+74.6%#5 of 38, top 14%EvalPlus

Agentic & Tool Use

BenchmarkScorePositionSettingSourceDate
TheAgentCompany3.4%#12 of 14, top 86%Epoch AI
Cybench7.5%#18 of 21, top 86%Epoch AI
BALROG21%#23 of 35, top 66%Epoch AI

Reasoning

BenchmarkScorePositionSettingSourceDate
ARC-AGI-20.8%#71 of 83, top 86%Epoch AI
SimpleBench27.1%#63 of 77, top 82%Epoch AI
LMArena Hard Prompts1255LMArena2026-10-08
LMArena Hard Prompts1296#191 of 297, top 65%LMArena2026-10-08
DTBench56.5%Epoch AI
DTBench59%#119 of 151, top 79%Epoch AI
BIG-Bench Hard89.2%Best of 27Epoch AI
BIG-Bench Hard84%Epoch AI
Epoch Capabilities Index131.73#135 of 213, top 64%Epoch AI2024-09-24
Epoch Capabilities Index126.91Epoch AI2024-05-14
ForecastBench58.4#49 of 72, top 69%Epoch AI

Math

BenchmarkScorePositionSettingSourceDate
OTIS Mock AIME 2024-20256.8%Epoch AI2025-02-25
OTIS Mock AIME 2024-202523.1%#123 of 173, top 72%Epoch AI2025-02-25
Omni-MATH36.4%#32 of 57, top 57%HELM Capabilities
LMArena Math1315#174 of 285, top 62%LMArena2026-10-08
LMArena Math1270LMArena2026-10-08
MATH Level 540.7%Epoch AI2025-01-27
MATH Level 570.4%#30 of 79, top 38%Epoch AI2025-01-27

Knowledge

BenchmarkScorePositionSettingSourceDate
GPQA Diamond45.9%Epoch AI2025-01-27
GPQA Diamond57.2%#116 of 186, top 63%Epoch AI2025-01-27
Humanity's Last Exam4.6%#36 of 41, top 88%Epoch AI
MMLU-Pro73.7%#29 of 58, top 50%HELM Capabilities
Confabulations (lower is better)13.5%#12 of 51, top 24%septLech Mazur benchmarks
GPQA (HELM)53.4%#29 of 57, top 51%HELM Capabilities
LMArena Expert1279#184 of 273, top 68%LMArena2026-10-08
LMArena Expert1245LMArena2026-10-08
MMLU86.9%#4 of 81, top 5%Epoch AI
MMLU82.7%Epoch AI
MMLU85.9%Epoch AI

Multimodal

BenchmarkScorePositionSettingSourceDate
LMArena Vision1161#95 of 122, top 78%LMArena2026-10-09
LMArena Vision1088LMArena2026-10-09
Video-MME75%#2 of 15, top 14%Epoch AI
Video-MME75%#2 of 15, top 14%Epoch AI

Multilingual

BenchmarkScorePositionSettingSourceDate
LMArena Non-English1264LMArena2026-10-08
LMArena Non-English1312#174 of 297, top 59%LMArena2026-10-08
LMArena Chinese1331#176 of 285, top 62%LMArena2026-10-08
LMArena Chinese1275LMArena2026-10-08
LMArena French1272LMArena2026-10-08
LMArena French1302#162 of 223, top 73%LMArena2026-10-08
LMArena German1286#156 of 231, top 68%LMArena2026-10-08
LMArena German1249LMArena2026-10-08
LMArena Japanese1292#125 of 211, top 60%LMArena2026-10-08
LMArena Japanese1239LMArena2026-10-08
LMArena Korean1298#133 of 213, top 63%LMArena2026-10-08
LMArena Korean1225LMArena2026-10-08
LMArena Russian1320#168 of 283, top 60%LMArena2026-10-08
LMArena Russian1276LMArena2026-10-08
LMArena Spanish1311#158 of 226, top 70%LMArena2026-10-08
LMArena Spanish1243LMArena2026-10-08

Instruction Following

BenchmarkScorePositionSettingSourceDate
IFEval83.7%#26 of 57, top 46%HELM Capabilities
LMArena Instruction Following1297#184 of 298, top 62%LMArena2026-10-08
LMArena Instruction Following1254LMArena2026-10-08

Long Context

BenchmarkScorePositionSettingSourceDate
LMArena Longer Query1291LMArena2026-10-08
LMArena Longer Query1308#183 of 291, top 63%LMArena2026-10-08

Writing & Preference

BenchmarkScorePositionSettingSourceDate
LMArena Text1319#181 of 297, top 61%LMArena2026-10-08
LMArena Text1274LMArena2026-10-08
LMArena Creative Writing1273LMArena2026-10-08
LMArena Creative Writing1333#143 of 295, top 49%LMArena2026-10-08
WildBench81.3%#24 of 57, top 43%HELM Capabilities
LMArena Multi-Turn1268LMArena2026-10-08
LMArena Multi-Turn1296#191 of 295, top 65%LMArena2026-10-08

Compare Gemini 1.5 Pro (May 2024)

Other Google models

Frequently asked questions

How good is Gemini 1.5 Pro (May 2024)?

Gemini 1.5 Pro (May 2024) by Google ranks 261st of 354 ranked models on the Noometry Index as of October 2026, with a score of 32.1. Its strongest category is multimodal, where it ranks 77th.

Is Gemini 1.5 Pro (May 2024) open source?

No. Gemini 1.5 Pro (May 2024) is proprietary and available only through Google's API and partner platforms.

What are Gemini 1.5 Pro (May 2024)'s strengths and weaknesses?

Relative to other ranked models, Gemini 1.5 Pro (May 2024) places best in writing & preference, long context, multilingual and lowest in reasoning, agentic & tool use, math.

What is Gemini 1.5 Pro (May 2024) best at?

Its best category is multimodal, where it ranks 77th on Noometry.