Noometry

Model comparison

GPT-4 vs Llama 2-13B

GPT-4 and Llama 2-13B score almost the same on the Noometry Index (29.1 vs 29.6), so choose on price, context window or the category you care about most.

Last verified . 26 shared benchmarks.

GPT-4 OpenAI

29.1

Rank #316 Confirmed

Llama 2-13B Meta

29.6

Rank #309 Confirmed

Summary

  • They share 26 benchmarks with published results for both. GPT-4 scores higher in 6 categories and Llama 2-13B in 2 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Llama 2-13B leads 31.1 to 10.8.
  • The biggest single-benchmark swing is DTBench: 62.7% for GPT-4 and 42.2% for Llama 2-13B.
  • Llama 2-13B has downloadable open weights; the other is API-only.

Side by side

GPT-4Llama 2-13B
ProviderOpenAIMeta
Noometry Index29.129.6
Released2023-03-142023-07-18
WeightsProprietaryOpen
Context window8K—
Max output8K—
Input $ / M tokens$30—
Output $ / M tokens$60—
Results tracked3832

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Too close to call

GPT-4: 31.6 (#283), Llama 2-13B: 30.9 (#291)

BenchmarkGPT-4Llama 2-13B
LMArena Coding12541062
WeirdML12.4%—
BigCodeBench Instruct46%—
BigCodeBench Complete57.2%—
HumanEval+79.3%—

Agentic & Tool Use Not comparable

GPT-4: —, Llama 2-13B: —

BenchmarkGPT-4Llama 2-13B
METR Time Horizons36.1%—

Reasoning GPT-4 leads

GPT-4: 17.8 (#289), Llama 2-13B: 12.8 (#337)

BenchmarkGPT-4Llama 2-13B
Chess Puzzles4%0%
LMArena Hard Prompts12411051
DTBench62.7%42.2%
BIG-Bench Hard75.1%58.2%
Epoch Capabilities Index125.89106.17
HellaSwag95.3%80.7%
WinoGrande87.5%72.8%
Mystery Game Puzzles12%—
LMCA17.1%—
ForecastBench57.8—
LAMBADA—76.5%
PIQA—80.8%

Math Llama 2-13B leads

GPT-4: 10.8 (#309), Llama 2-13B: 31.1 (#229)

BenchmarkGPT-4Llama 2-13B
LMArena Math12691065
GSM8K92%36.9%
OTIS Mock AIME 2024-20251.1%—
MATH Level 523%—

Knowledge Llama 2-13B leads

GPT-4: 18.4 (#282), Llama 2-13B: 28.1 (#249)

BenchmarkGPT-4Llama 2-13B
LMArena Expert12111030
MMLU86.4%55.6%
TriviaQA84.8%79.6%
GPQA Diamond35.7%—
ARC (AI2) Challenge—60.3%
BoolQ—82.4%
OpenBookQA—57%

Multimodal Not comparable

GPT-4: —, Llama 2-13B: —

BenchmarkGPT-4Llama 2-13B
ScienceQA—55.8%

Multilingual GPT-4 leads

GPT-4: 40.6 (#215), Llama 2-13B: 26.5 (#279)

BenchmarkGPT-4Llama 2-13B
LMArena Non-English12461024
LMArena Chinese12421001
LMArena French12831044
LMArena German12511009
LMArena Japanese1209894
LMArena Korean1184953
LMArena Russian12511055
LMArena Spanish12611087

Instruction Following GPT-4 leads

GPT-4: 65.3 (#222), Llama 2-13B: 53.3 (#287)

BenchmarkGPT-4Llama 2-13B
LMArena Instruction Following12411045

Long Context GPT-4 leads

GPT-4: 37.7 (#212), Llama 2-13B: 32.3 (#269)

BenchmarkGPT-4Llama 2-13B
LMArena Longer Query12441064

Writing & Preference GPT-4 leads

GPT-4: 34.9 (#268), Llama 2-13B: 29.8 (#289)

BenchmarkGPT-4Llama 2-13B
LMArena Text12631084
LMArena Creative Writing12441047
LMArena Multi-Turn12571050
EQ-Bench Creative Writing752—

Frequently asked questions

Is GPT-4 better than Llama 2-13B?

GPT-4 and Llama 2-13B score almost the same on the Noometry Index (29.1 vs 29.6), so choose on price, context window or the category you care about most.

Is GPT-4 or Llama 2-13B better for coding?

They score almost the same on coding (31.6 vs 30.9); test both on your own repository before choosing.

How many benchmarks do GPT-4 and Llama 2-13B share?

26 benchmarks have published results for both models. GPT-4 has 38 scored results on Noometry and Llama 2-13B has 32.

Related comparisons

Go deeper