Noometry

Model comparison

Mistral Nemo vs Yi-34B

Yi-34B is the stronger model overall, scoring 27.4 to 25.9 on the Noometry Index.

Last verified . 4 shared benchmarks.

Mistral Nemo Mistral AI

25.9

Rank #332 Confirmed

Yi-34B 01.AI

27.4

Rank #325 Confirmed

Summary

  • They share 4 benchmarks with published results for both. Mistral Nemo scores higher in 2 categories and Yi-34B in 2 categories; 3 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Yi-34B leads 34.3 to 28.6.
  • The biggest single-benchmark swing is GPQA Diamond: 29.9% for Mistral Nemo and 14.7% for Yi-34B.

Side by side

Mistral NemoYi-34B
ProviderMistral AI01.AI
Noometry Index25.927.4
Released2024-07-012023-11-02
WeightsOpenOpen
Context window128K—
Max output128K—
Input $ / M tokens$0.15—
Output $ / M tokens$0.15—
Results tracked1023

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Mistral Nemo: —, Yi-34B: 31.8 (#269)

BenchmarkMistral NemoYi-34B
LMArena Coding—1112

Agentic & Tool Use Not comparable

Mistral Nemo: 23.3 (#118), Yi-34B: —

BenchmarkMistral NemoYi-34B
Berkeley Function Calling Leaderboard27.6%—
BALROG17.6%—

Reasoning Too close to call

Mistral Nemo: 19.9 (#230), Yi-34B: 20.3 (#222)

BenchmarkMistral NemoYi-34B
Epoch Capabilities Index118.63117.28
LMArena Hard Prompts—1104
DTBench48.6%—
BIG-Bench Hard—71.7%
PIQA83.5%—

Math Mistral Nemo leads

Mistral Nemo: 23.5 (#263), Yi-34B: 19.8 (#277)

BenchmarkMistral NemoYi-34B
MATH Level 510.8%5.1%
GSM8K84.2%76%
LMArena Math—1114

Knowledge Mistral Nemo leads

Mistral Nemo: 12.3 (#293), Yi-34B: 7.5 (#304)

BenchmarkMistral NemoYi-34B
GPQA Diamond29.9%14.7%
LMArena Expert—1061
BoolQ82.5%—
MMLU—76.3%

Multilingual Not comparable

Mistral Nemo: —, Yi-34B: 29.9 (#259)

BenchmarkMistral NemoYi-34B
LMArena Non-English—1079
LMArena Chinese—1175
LMArena French—1080
LMArena German—1042
LMArena Japanese—993
LMArena Korean—959
LMArena Russian—1050
LMArena Spanish—1070

Instruction Following Not comparable

Mistral Nemo: —, Yi-34B: 56.4 (#269)

BenchmarkMistral NemoYi-34B
LMArena Instruction Following—1091

Long Context Not comparable

Mistral Nemo: —, Yi-34B: 33.2 (#259)

BenchmarkMistral NemoYi-34B
LMArena Longer Query—1094

Writing & Preference Yi-34B leads

Mistral Nemo: 28.6 (#291), Yi-34B: 34.3 (#268)

BenchmarkMistral NemoYi-34B
LMArena Text—1129
LMArena Creative Writing—1108
EQ-Bench Creative Writing881—
LMArena Multi-Turn—1113

Frequently asked questions

Is Mistral Nemo better than Yi-34B?

Yi-34B is the stronger model overall, scoring 27.4 to 25.9 on the Noometry Index.

How many benchmarks do Mistral Nemo and Yi-34B share?

4 benchmarks have published results for both models. Mistral Nemo has 10 scored results on Noometry and Yi-34B has 23.

Related comparisons

Go deeper