Noometry

Model comparison

GPT-4 vs Nemotron 3 Super

Nemotron 3 Super is the stronger model overall, scoring 40.1 to 29.1 on the Noometry Index.

Last verified . 16 shared benchmarks.

GPT-4 OpenAI

29.1

Rank #316 Confirmed

Nemotron 3 Super NVIDIA

40.1

Rank #153 Confirmed

Summary

  • They share 16 benchmarks with published results for both. GPT-4 scores higher in 0 categories and Nemotron 3 Super in 8 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Nemotron 3 Super leads 39.6 to 10.8.
  • The biggest single-benchmark swing is WeirdML: 12.4% for GPT-4 and 38% for Nemotron 3 Super.
  • Nemotron 3 Super is cheaper at $0.08 / $0.45 per million input/output tokens, against $30 / $60 for GPT-4.
  • Nemotron 3 Super accepts more context: 262K tokens versus 8K.
  • Nemotron 3 Super has downloadable open weights; the other is API-only.

Side by side

GPT-4Nemotron 3 Super
ProviderOpenAINVIDIA
Noometry Index29.140.1
Released2023-03-142026-03-11
WeightsProprietaryOpen
Context window8K262K
Max output8K131K
Input $ / M tokens$30$0.08
Output $ / M tokens$60$0.45
Results tracked3821

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Nemotron 3 Super leads

GPT-4: 31.6 (#283), Nemotron 3 Super: 39.4 (#158)

BenchmarkGPT-4Nemotron 3 Super
WeirdML12.4%38%
LMArena Coding12541403
SciCode—36%
BigCodeBench Instruct46%—
BigCodeBench Complete57.2%—
ALE-Bench—213.9
HumanEval+79.3%—

Agentic & Tool Use Not comparable

GPT-4: —, Nemotron 3 Super: —

BenchmarkGPT-4Nemotron 3 Super
METR Time Horizons36.1%—

Reasoning Too close to call

GPT-4: 17.8 (#289), Nemotron 3 Super: 18.7 (#276)

BenchmarkGPT-4Nemotron 3 Super
LMArena Hard Prompts12411386
NYT Connections (extended)—15.4%
CritPt—3.1%
Chess Puzzles4%—
Mystery Game Puzzles12%—
DTBench62.7%—
LMCA17.1%—
BIG-Bench Hard75.1%—
Epoch Capabilities Index125.89—
ForecastBench57.8—
HellaSwag95.3%—
WinoGrande87.5%—

Math Nemotron 3 Super leads

GPT-4: 10.8 (#309), Nemotron 3 Super: 39.6 (#101)

BenchmarkGPT-4Nemotron 3 Super
LMArena Math12691380
MathArena Final-Answer Competitions—60.4%
OTIS Mock AIME 2024-20251.1%—
MATH Level 523%—
GSM8K92%—

Knowledge Nemotron 3 Super leads

GPT-4: 18.4 (#282), Nemotron 3 Super: 38.9 (#139)

BenchmarkGPT-4Nemotron 3 Super
LMArena Expert12111398
GPQA Diamond35.7%—
MMLU86.4%—
TriviaQA84.8%—

Multilingual Nemotron 3 Super leads

GPT-4: 40.6 (#215), Nemotron 3 Super: 48.4 (#144)

BenchmarkGPT-4Nemotron 3 Super
LMArena Non-English12461355
LMArena Chinese12421432
LMArena French12831405
LMArena German12511341
LMArena Russian12511336
LMArena Spanish12611417
LMArena Japanese1209—
LMArena Korean1184—

Instruction Following Nemotron 3 Super leads

GPT-4: 65.3 (#222), Nemotron 3 Super: 71.2 (#154)

BenchmarkGPT-4Nemotron 3 Super
LMArena Instruction Following12411347

Long Context Nemotron 3 Super leads

GPT-4: 37.7 (#212), Nemotron 3 Super: 41.5 (#139)

BenchmarkGPT-4Nemotron 3 Super
LMArena Longer Query12441362

Writing & Preference Nemotron 3 Super leads

GPT-4: 34.9 (#268), Nemotron 3 Super: 55.9 (#140)

BenchmarkGPT-4Nemotron 3 Super
LMArena Text12631378
LMArena Creative Writing12441317
LMArena Multi-Turn12571369
EQ-Bench Creative Writing752—

Frequently asked questions

Is GPT-4 better than Nemotron 3 Super?

Nemotron 3 Super is the stronger model overall, scoring 40.1 to 29.1 on the Noometry Index.

Which is cheaper, GPT-4 or Nemotron 3 Super?

Nemotron 3 Super is cheaper. It lists at $0.08 per million input tokens and $0.45 per million output tokens; GPT-4 lists at $30 and $60.

Is GPT-4 or Nemotron 3 Super better for coding?

Nemotron 3 Super scores higher on coding benchmarks: 39.4 versus 31.6 in the Noometry coding category.

Which has the bigger context window?

Nemotron 3 Super does, with 262K tokens against 8K.

How many benchmarks do GPT-4 and Nemotron 3 Super share?

16 benchmarks have published results for both models. GPT-4 has 38 scored results on Noometry and Nemotron 3 Super has 21.

Related comparisons

Go deeper