Noometry

Model comparison

Llama2 70b Steerlm Chat vs Mistral Small

Mistral Small is the stronger model overall, scoring 33.1 to 31.2 on the Noometry Index.

Last verified . 9 shared benchmarks.

Llama2 70b Steerlm Chat NVIDIA

31.2

Rank #265 Confirmed

Mistral Small Mistral AI

33.1

Rank #238 Confirmed

Summary

  • They share 9 benchmarks with published results for both. Llama2 70b Steerlm Chat scores higher in 1 category and Mistral Small in 6 categories; 6 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Mistral Small leads 52.6 to 31.8.

Side by side

Llama2 70b Steerlm ChatMistral Small
ProviderNVIDIAMistral AI
Noometry Index31.233.1
Released—2024-02-26
WeightsOpenOpen
Context window—256K
Max output—256K
Input $ / M tokens—$0.15
Output $ / M tokens—$0.60
Results tracked939

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mistral Small leads

Llama2 70b Steerlm Chat: 29.3 (#295), Mistral Small: 33.5 (#242)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Coding10251363
SciCode—26.5%
BigCodeBench Instruct—36.1%
LiveBench Coding—36.2%
BigCodeBench Complete—46.6%
ALE-Bench—497.62

Agentic & Tool Use Not comparable

Llama2 70b Steerlm Chat: —, Mistral Small: 27.9 (#87)

BenchmarkLlama2 70b Steerlm ChatMistral Small
Berkeley Function Calling Leaderboard—37.1%

Reasoning Too close to call

Llama2 70b Steerlm Chat: 19.1 (#245), Mistral Small: 19.3 (#238)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Hard Prompts10471336
Kagi LLM Benchmark—37.8%
CritPt—0%
LiveBench Reasoning—44.8%
DTBench—70.9%
LiveBench Data Analysis—53.7%
LMCA—20.6%
LiveBench—44%

Math Llama2 70b Steerlm Chat leads

Llama2 70b Steerlm Chat: 28.8 (#223), Mistral Small: 15.4 (#287)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Math10721341
OTIS Mock AIME 2024-2025—5.8%
LiveBench Math—39.9%
MATH Level 5—46.8%

Knowledge Not comparable

Llama2 70b Steerlm Chat: —, Mistral Small: 30.9 (#214)

BenchmarkLlama2 70b Steerlm ChatMistral Small
GPQA Diamond—47.5%
Vectara Hallucination Rate—5.1%
LMArena Expert—1293
MMLU—68.7%

Multimodal Not comparable

Llama2 70b Steerlm Chat: —, Mistral Small: 32.9 (#91)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Vision—1142

Multilingual Mistral Small leads

Llama2 70b Steerlm Chat: 28.9 (#265), Mistral Small: 45.7 (#164)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Non-English10641315
LMArena Chinese—1342
LMArena French—1337
LMArena German—1341
LMArena Japanese—1275
LMArena Korean—1260
LMArena Russian—1324
LMArena Spanish—1346

Instruction Following Mistral Small leads

Llama2 70b Steerlm Chat: 54.5 (#274), Mistral Small: 66.6 (#204)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Instruction Following10611310
LiveBench Instruction Following—63.7%

Long Context Mistral Small leads

Llama2 70b Steerlm Chat: 30.4 (#283), Mistral Small: 40.4 (#151)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Longer Query9981327

Writing & Preference Mistral Small leads

Llama2 70b Steerlm Chat: 31.8 (#278), Mistral Small: 52.6 (#165)

BenchmarkLlama2 70b Steerlm ChatMistral Small
LMArena Text10981338
LMArena Creative Writing10911304
LMArena Multi-Turn10581344
LiveBench Language—30.5%

Frequently asked questions

Is Llama2 70b Steerlm Chat better than Mistral Small?

Mistral Small is the stronger model overall, scoring 33.1 to 31.2 on the Noometry Index.

Is Llama2 70b Steerlm Chat or Mistral Small better for coding?

Mistral Small scores higher on coding benchmarks: 33.5 versus 29.3 in the Noometry coding category.

How many benchmarks do Llama2 70b Steerlm Chat and Mistral Small share?

9 benchmarks have published results for both models. Llama2 70b Steerlm Chat has 9 scored results on Noometry and Mistral Small has 39.

Related comparisons

Go deeper