Model comparison

Gemma 3 1B vs GPT-5.4

GPT-5.4 is the stronger model overall, scoring 59.4 to 21.1 on the Noometry Index.

Last verified October 10, 2026. 3 shared benchmarks.

Gemma 3 1B

Google

21.1

Rank #353 Reported

GPT-5.4 OpenAI

59.4

Rank #16 Confirmed

Summary

They share 3 benchmarks with published results for both. Gemma 3 1B scores higher in 0 categories and GPT-5.4 in 4 categories; 4 gaps are clear of the uncertainty.
The widest gap is in math, where GPT-5.4 leads 73.5 to 10.2.
The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 1.1% for Gemma 3 1B and 97.8% for GPT-5.4.
Gemma 3 1B has downloadable open weights; the other is API-only.

Side by side

Gemma 3 1B and GPT-5.4 specifications
	Gemma 3 1B	GPT-5.4
Provider	Google	OpenAI
Noometry Index	21.1	59.4
Released	2025-03-12	2026-03-05
Weights	Open	Proprietary
Context window	—	1.05M
Max output	—	128K
Input $ / M tokens	—	$2.50
Output $ / M tokens	—	$15
Results tracked	4	68

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Gemma 3 1B: —, GPT-5.4: 52.6 (#33)

Coding benchmarks
Benchmark	Gemma 3 1B	GPT-5.4
SWE-bench Verified	—	76.9%
DeepSWE	—	51.8%
LMArena WebDev	—	1465
SciCode	—	56.6%
GSO	—	31.4%
WeirdML	—	77.7%
LMArena Coding	—	1497
MirrorCode	—	15.6%
ALE-Bench	—	1,607
AlgoTune	—	1.85

Agentic & Tool Use GPT-5.4 leads

Gemma 3 1B: 13.7 (#152), GPT-5.4: 46.5 (#13)

Agentic & Tool Use benchmarks
Benchmark	Gemma 3 1B	GPT-5.4
Terminal-Bench	—	81.8%
APEX-Agents	—	52.4%
Berkeley Function Calling Leaderboard	7.2%	—
τ²-bench Banking	—	39.4%
DeepResearch Bench	—	35.1%
PostTrainBench	—	19%
GBAEval	—	45.1%
LMArena Search	—	1197
METR Time Horizons	—	74.3%
Vending-Bench 2	—	6,144

Reasoning GPT-5.4 leads

Gemma 3 1B: 19.2 (#264), GPT-5.4: 61.8 (#19)

Reasoning benchmarks
Benchmark	Gemma 3 1B	GPT-5.4
Chess Puzzles	0%	44%
ARC-AGI-2	—	74%
Kagi LLM Benchmark	—	63.8%
NYT Connections (extended)	—	91.3%
ARC-AGI-1	—	93.7%
CritPt	—	23.4%
EnigmaEval	—	16%
Thematic Generalization	—	80%
EBR-Bench	—	25.4%
LMArena Hard Prompts	—	1485
Mystery Game Puzzles	—	37%
DTBench	—	94.4%
LMCA	—	52%
Epoch Capabilities Index	—	156.81
ForecastBench	—	59.5

Math GPT-5.4 leads

Gemma 3 1B: 10.2 (#316), GPT-5.4: 73.5 (#19)

Math benchmarks
Benchmark	Gemma 3 1B	GPT-5.4
OTIS Mock AIME 2024-2025	1.1%	97.8%
FrontierMath (Tiers 1-3)	—	78.6%
FrontierMath Tier 4	—	49%
MathArena Final-Answer Competitions	—	83.1%
ProofBench	—	56%
LMArena Math	—	1488
FrontierMath (Feb 2025 set)	—	47.6%
FrontierMath Tier 4 (v1)	—	27.1%

Knowledge GPT-5.4 leads

Gemma 3 1B: 7.0 (#314), GPT-5.4: 65.3 (#14)

Knowledge benchmarks
Benchmark	Gemma 3 1B	GPT-5.4
GPQA Diamond	19.9%	93.3%
Humanity's Last Exam	—	36.2%
SimpleQA Verified	—	45.1%
Vectara Hallucination Rate	—	7%
LMArena Expert	—	1507

Multimodal Not comparable

Gemma 3 1B: —, GPT-5.4: 43.7 (#20)

Multimodal benchmarks
Benchmark	Gemma 3 1B	GPT-5.4
LMArena Vision	—	1303
Blueprint-Bench 2	—	27.1%
Furniture Assembly	—	37.5%
LMArena Document	—	1471

Multilingual Not comparable

Gemma 3 1B: —, GPT-5.4: 56.2 (#23)

Multilingual benchmarks
Benchmark	Gemma 3 1B	GPT-5.4
LMArena Non-English	—	1465
LMArena Chinese	—	1519
LMArena French	—	1493
LMArena German	—	1472
LMArena Japanese	—	1485
LMArena Korean	—	1448
LMArena Russian	—	1480
LMArena Spanish	—	1454

Instruction Following Not comparable

Gemma 3 1B: —, GPT-5.4: 77.1 (#27)

Instruction Following benchmarks
Benchmark	Gemma 3 1B	GPT-5.4
LMArena Instruction Following	—	1469

Long Context Not comparable

Gemma 3 1B: —, GPT-5.4: 50.3 (#8)

Long Context benchmarks
Benchmark	Gemma 3 1B	GPT-5.4
CL-bench	—	27.9%
CL-bench Life	—	21.7%
LMArena Longer Query	—	1473

Writing & Preference Not comparable

Gemma 3 1B: —, GPT-5.4: 71.9 (#17)

Writing & Preference benchmarks
Benchmark	Gemma 3 1B	GPT-5.4
LMArena Text	—	1469
LMArena Creative Writing	—	1439
EQ-Bench Creative Writing	—	1840
EQ-Bench 4	—	1272
LMArena Multi-Turn	—	1482

Frequently asked questions

Is Gemma 3 1B better than GPT-5.4?