Noometry

Model comparison

GPT-4 Turbo vs Yi-34B

GPT-4 Turbo is the stronger model overall, scoring 30.5 to 27.8 on the Noometry Index.

Last verified . 21 shared benchmarks.

GPT-4 Turbo OpenAI

30.5

Rank #292 Confirmed

Yi-34B 01.AI

27.8

Rank #329 Confirmed

Summary

  • They share 21 benchmarks with published results for both. GPT-4 Turbo scores higher in 6 categories and Yi-34B in 2 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where GPT-4 Turbo leads 24.3 to 7.5.
  • The biggest single-benchmark swing is MATH Level 5: 46.7% for GPT-4 Turbo and 5.1% for Yi-34B.
  • Yi-34B has downloadable open weights; the other is API-only.

Side by side

GPT-4 TurboYi-34B
ProviderOpenAI01.AI
Noometry Index30.527.8
Released2023-11-062023-11-02
WeightsProprietaryOpen
Context window128K—
Max output4K—
Input $ / M tokens$10—
Output $ / M tokens$30—
Results tracked3623

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4 Turbo leads

GPT-4 Turbo: 33.8 (#249), Yi-34B: 32.3 (#274)

BenchmarkGPT-4 TurboYi-34B
LMArena Coding12681112
WeirdML18%—
BigCodeBench Instruct48.2%—
BigCodeBench Complete58.2%—
HumanEval+86.6%—
MBPP+73.3%—

Agentic & Tool Use Not comparable

GPT-4 Turbo: —, Yi-34B: —

BenchmarkGPT-4 TurboYi-34B
METR Time Horizons36.7%—

Reasoning Yi-34B leads

GPT-4 Turbo: 15.3 (#317), Yi-34B: 21.2 (#226)

BenchmarkGPT-4 TurboYi-34B
LMArena Hard Prompts12511104
Epoch Capabilities Index127.25117.39
SimpleBench25.1%—
Chess Puzzles6%—
DTBench61.6%—
LMCA9.8%—
BIG-Bench Hard—71.7%
ForecastBench59.4—

Math Yi-34B leads

GPT-4 Turbo: 9.0 (#322), Yi-34B: 21.6 (#282)

BenchmarkGPT-4 TurboYi-34B
LMArena Math12721114
MATH Level 546.7%5.1%
FrontierMath (Tiers 1-3)0.7%—
OTIS Mock AIME 2024-20256.7%—
GSM8K—76%

Knowledge GPT-4 Turbo leads

GPT-4 Turbo: 24.3 (#268), Yi-34B: 7.5 (#309)

BenchmarkGPT-4 TurboYi-34B
GPQA Diamond46.6%14.7%
LMArena Expert12231061
MMLU81.3%76.3%
Confabulations28.4%—

Multimodal Not comparable

GPT-4 Turbo: 30.6 (#110), Yi-34B: —

BenchmarkGPT-4 TurboYi-34B
LMArena Vision1090—

Multilingual GPT-4 Turbo leads

GPT-4 Turbo: 40.5 (#216), Yi-34B: 29.7 (#264)

BenchmarkGPT-4 TurboYi-34B
LMArena Non-English12451079
LMArena Chinese12421176
LMArena French12761081
LMArena German12591042
LMArena Japanese1194993
LMArena Korean1187959
LMArena Russian12591050
LMArena Spanish12601070

Instruction Following GPT-4 Turbo leads

GPT-4 Turbo: 65.8 (#216), Yi-34B: 56.2 (#274)

BenchmarkGPT-4 TurboYi-34B
LMArena Instruction Following12491091

Long Context GPT-4 Turbo leads

GPT-4 Turbo: 38.0 (#206), Yi-34B: 33.2 (#264)

BenchmarkGPT-4 TurboYi-34B
LMArena Longer Query12541094

Writing & Preference GPT-4 Turbo leads

GPT-4 Turbo: 47.7 (#206), Yi-34B: 34.1 (#273)

BenchmarkGPT-4 TurboYi-34B
LMArena Text12721129
LMArena Creative Writing12691108
LMArena Multi-Turn12671113

Frequently asked questions

Is GPT-4 Turbo better than Yi-34B?

GPT-4 Turbo is the stronger model overall, scoring 30.5 to 27.8 on the Noometry Index.

Is GPT-4 Turbo or Yi-34B better for coding?

GPT-4 Turbo scores higher on coding benchmarks: 33.8 versus 32.3 in the Noometry coding category.

How many benchmarks do GPT-4 Turbo and Yi-34B share?

21 benchmarks have published results for both models. GPT-4 Turbo has 36 scored results on Noometry and Yi-34B has 23.

Related comparisons

Go deeper