Noometry

Model comparison

Llama 4 Scout vs Mistral Nemo

Llama 4 Scout is the stronger model overall, scoring 27.4 to 25.9 on the Noometry Index.

Last verified . 6 shared benchmarks.

Llama 4 Scout Meta

27.4

Rank #323 Confirmed

Mistral Nemo Mistral AI

25.9

Rank #332 Confirmed

Summary

  • They share 6 benchmarks with published results for both. Llama 4 Scout scores higher in 3 categories and Mistral Nemo in 2 categories; 5 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Llama 4 Scout leads 31.8 to 12.3.
  • The biggest single-benchmark swing is MATH Level 5: 62.3% for Llama 4 Scout and 10.8% for Mistral Nemo.
  • Both cost about the same: $0.10 input and $0.30 output per million tokens.

Side by side

Llama 4 ScoutMistral Nemo
ProviderMetaMistral AI
Noometry Index27.425.9
Released2025-04-052024-07-01
WeightsOpenOpen
Context window128K128K
Max output4K128K
Input $ / M tokens$0.10$0.15
Output $ / M tokens$0.30$0.15
Results tracked4310

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Llama 4 Scout: 19.9 (#334), Mistral Nemo: —

BenchmarkLlama 4 ScoutMistral Nemo
SWE-bench Verified (bash only)9.1%—
SciCode17%—
LMArena Coding1286—
BigCodeBench Complete43.1%—

Agentic & Tool Use Llama 4 Scout leads

Llama 4 Scout: 24.4 (#109), Mistral Nemo: 23.3 (#118)

BenchmarkLlama 4 ScoutMistral Nemo
Berkeley Function Calling Leaderboard28.1%27.6%
BALROG—17.6%

Reasoning Mistral Nemo leads

Llama 4 Scout: 9.2 (#339), Mistral Nemo: 19.9 (#230)

BenchmarkLlama 4 ScoutMistral Nemo
DTBench57.9%48.6%
Epoch Capabilities Index129.64118.63
ARC-AGI-20%—
Kagi LLM Benchmark36.9%—
ARC-AGI-10.5%—
CritPt0%—
LMArena Hard Prompts1266—
LMCA12%—
ForecastBench57.5—
PIQA—83.5%

Math Mistral Nemo leads

Llama 4 Scout: 18.4 (#280), Mistral Nemo: 23.5 (#263)

BenchmarkLlama 4 ScoutMistral Nemo
MATH Level 562.3%10.8%
OTIS Mock AIME 2024-20257.8%—
Omni-MATH37.3%—
LMArena Math1286—
FrontierMath (Feb 2025 set)0%—
GSM8K—84.2%

Knowledge Llama 4 Scout leads

Llama 4 Scout: 31.8 (#208), Mistral Nemo: 12.3 (#293)

BenchmarkLlama 4 ScoutMistral Nemo
GPQA Diamond51.8%29.9%
MMLU-Pro74.2%—
Vectara Hallucination Rate7.7%—
GPQA (HELM)50.7%—
LMArena Expert1236—
BoolQ—82.5%

Multimodal Not comparable

Llama 4 Scout: 31.6 (#98), Mistral Nemo: —

BenchmarkLlama 4 ScoutMistral Nemo
LMArena Vision1118—
SpatialViz-Bench34.2%—

Multilingual Not comparable

Llama 4 Scout: 41.3 (#207), Mistral Nemo: —

BenchmarkLlama 4 ScoutMistral Nemo
LMArena Non-English1253—
LMArena Chinese1256—
LMArena French1283—
LMArena German1274—
LMArena Japanese1209—
LMArena Korean1207—
LMArena Russian1263—
LMArena Spanish1277—

Instruction Following Not comparable

Llama 4 Scout: 66.0 (#212), Mistral Nemo: —

BenchmarkLlama 4 ScoutMistral Nemo
IFEval81.8%—
LMArena Instruction Following1248—

Long Context Not comparable

Llama 4 Scout: 27.5 (#289), Mistral Nemo: —

BenchmarkLlama 4 ScoutMistral Nemo
Fiction.LiveBench36%—
LMArena Longer Query1265—

Writing & Preference Llama 4 Scout leads

Llama 4 Scout: 37.1 (#256), Mistral Nemo: 28.6 (#291)

BenchmarkLlama 4 ScoutMistral Nemo
EQ-Bench Creative Writing783881
LMArena Text1279—
LMArena Creative Writing1249—
WildBench78%—
LMArena Multi-Turn1281—

Frequently asked questions

Is Llama 4 Scout better than Mistral Nemo?

Llama 4 Scout is the stronger model overall, scoring 27.4 to 25.9 on the Noometry Index.

Which is cheaper, Llama 4 Scout or Mistral Nemo?

Mistral Nemo is cheaper. It lists at $0.15 per million input tokens and $0.15 per million output tokens; Llama 4 Scout lists at $0.10 and $0.30.

Which has the bigger context window?

Both accept 128K tokens.

How many benchmarks do Llama 4 Scout and Mistral Nemo share?

6 benchmarks have published results for both models. Llama 4 Scout has 43 scored results on Noometry and Mistral Nemo has 10.

Related comparisons

Go deeper