Noometry

Model comparison

Llama2 70b Steerlm Chat vs Mistral Small 3.1

Llama2 70b Steerlm Chat and Mistral Small 3.1 score almost the same on the Noometry Index (31.8 vs 31.7), so choose on price, context window or the category you care about most.

Last verified . 9 shared benchmarks.

Llama2 70b Steerlm Chat NVIDIA

31.8

Rank #268 Confirmed

Mistral Small 3.1 Mistral AI

31.7

Rank #269 Confirmed

Summary

  • They share 9 benchmarks with published results for both. Llama2 70b Steerlm Chat scores higher in 2 categories and Mistral Small 3.1 in 5 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in math, where Llama2 70b Steerlm Chat leads 31.3 to 14.7.

Side by side

Llama2 70b Steerlm ChatMistral Small 3.1
ProviderNVIDIAMistral AI
Noometry Index31.831.7
Released—2025-03-17
WeightsOpenOpen
Context window—128K
Max output—102K
Input $ / M tokens—$0.35
Output $ / M tokens—$0.56
Results tracked928

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Small 3.1 leads

Llama2 70b Steerlm Chat: 29.9 (#300), Mistral Small 3.1: 38.3 (#179)

BenchmarkLlama2 70b Steerlm ChatMistral Small 3.1
LMArena Coding10251309

Reasoning Too close to call

Llama2 70b Steerlm Chat: 20.0 (#246), Mistral Small 3.1: 19.7 (#254)

BenchmarkLlama2 70b Steerlm ChatMistral Small 3.1
LMArena Hard Prompts10471278
Chess Puzzles—1%
Epoch Capabilities Index—127.48

Math Llama2 70b Steerlm Chat leads

Llama2 70b Steerlm Chat: 31.3 (#226), Mistral Small 3.1: 14.7 (#301)

BenchmarkLlama2 70b Steerlm ChatMistral Small 3.1
LMArena Math10721262
OTIS Mock AIME 2024-2025—3.9%
Omni-MATH—24.8%

Knowledge Not comparable

Llama2 70b Steerlm Chat: —, Mistral Small 3.1: 22.6 (#271)

BenchmarkLlama2 70b Steerlm ChatMistral Small 3.1
GPQA Diamond—41.9%
MMLU-Pro—61%
GPQA (HELM)—39.2%
LMArena Expert—1257

Multimodal Not comparable

Llama2 70b Steerlm Chat: —, Mistral Small 3.1: 33.2 (#99)

BenchmarkLlama2 70b Steerlm ChatMistral Small 3.1
LMArena Vision—1136

Multilingual Mistral Small 3.1 leads

Llama2 70b Steerlm Chat: 28.8 (#270), Mistral Small 3.1: 41.2 (#209)

BenchmarkLlama2 70b Steerlm ChatMistral Small 3.1
LMArena Non-English10631255
LMArena Chinese—1253
LMArena French—1273
LMArena German—1266
LMArena Japanese—1208
LMArena Korean—1206
LMArena Russian—1263
LMArena Spanish—1283

Instruction Following Mistral Small 3.1 leads

Llama2 70b Steerlm Chat: 54.2 (#279), Mistral Small 3.1: 63.6 (#230)

BenchmarkLlama2 70b Steerlm ChatMistral Small 3.1
LMArena Instruction Following10601264
IFEval—75%

Long Context Mistral Small 3.1 leads

Llama2 70b Steerlm Chat: 30.4 (#288), Mistral Small 3.1: 39.5 (#178)

BenchmarkLlama2 70b Steerlm ChatMistral Small 3.1
LMArena Longer Query9981299

Writing & Preference Mistral Small 3.1 leads

Llama2 70b Steerlm Chat: 31.6 (#283), Mistral Small 3.1: 37.0 (#259)

BenchmarkLlama2 70b Steerlm ChatMistral Small 3.1
LMArena Text10981277
LMArena Creative Writing10911253
LMArena Multi-Turn10581270
EQ-Bench Creative Writing—761
WildBench—78.8%

Frequently asked questions

Is Llama2 70b Steerlm Chat better than Mistral Small 3.1?

Llama2 70b Steerlm Chat and Mistral Small 3.1 score almost the same on the Noometry Index (31.8 vs 31.7), so choose on price, context window or the category you care about most.

Is Llama2 70b Steerlm Chat or Mistral Small 3.1 better for coding?

Mistral Small 3.1 scores higher on coding benchmarks: 38.3 versus 29.9 in the Noometry coding category.

How many benchmarks do Llama2 70b Steerlm Chat and Mistral Small 3.1 share?

9 benchmarks have published results for both models. Llama2 70b Steerlm Chat has 9 scored results on Noometry and Mistral Small 3.1 has 28.

Related comparisons

Go deeper