Model comparison

Llama 3.2 90B vs Mixtral 8x7B

Llama 3.2 90B and Mixtral 8x7B score almost the same on the Noometry Index (27.5 vs 27.1), so choose on price, context window or the category you care about most.

Last verified . 4 shared benchmarks.

Llama 3.2 90B Meta

27.5

Rank #331 Confirmed

Mixtral 8x7B Mistral AI

27.1

Rank #334 Confirmed

Summary

  • They share 4 benchmarks with published results for both. Llama 3.2 90B scores higher in 2 categories and Mixtral 8x7B in 1 category; 3 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Llama 3.2 90B leads 21.7 to 11.0.
  • The biggest single-benchmark swing is MATH Level 5: 39.4% for Llama 3.2 90B and 10% for Mixtral 8x7B.

Side by side

Llama 3.2 90B and Mixtral 8x7B specifications
Llama 3.2 90BMixtral 8x7B
ProviderMetaMistral AI
Noometry Index27.527.1
Released2024-09-242023-12-11
WeightsOpenOpen
Context window—32K
Max output—32K
Input $ / M tokens—$0.70
Output $ / M tokens—$0.70
Results tracked938

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Llama 3.2 90B: —, Mixtral 8x7B: 32.8 (#269)

Coding benchmarks
BenchmarkLlama 3.2 90BMixtral 8x7B
LMArena Coding—1126
HumanEval+—39.6%
MBPP+—49.7%

Agentic & Tool Use Not comparable

Llama 3.2 90B: 30.0 (#80), Mixtral 8x7B: —

Agentic & Tool Use benchmarks
BenchmarkLlama 3.2 90BMixtral 8x7B
BALROG27.3%—

Reasoning Llama 3.2 90B leads

Llama 3.2 90B: 21.7 (#217), Mixtral 8x7B: 18.2 (#285)

Reasoning benchmarks
BenchmarkLlama 3.2 90BMixtral 8x7B
Epoch Capabilities Index125.5118.47
EnigmaEval0.4%—
LMArena Hard Prompts—1115
DTBench—49.6%
Adversarial NLI—55.2%
ForecastBench—56.3
HellaSwag—86.7%
PIQA—83.6%
WinoGrande—77.2%

Math Mixtral 8x7B leads

Llama 3.2 90B: 11.1 (#308), Mixtral 8x7B: 18.8 (#289)

Math benchmarks
BenchmarkLlama 3.2 90BMixtral 8x7B
MATH Level 539.4%10%
OTIS Mock AIME 2024-20252.6%—
Omni-MATH—10.5%
LMArena Math—1147
GSM8K—74.4%

Knowledge Llama 3.2 90B leads

Llama 3.2 90B: 21.7 (#274), Mixtral 8x7B: 11.0 (#301)

Knowledge benchmarks
BenchmarkLlama 3.2 90BMixtral 8x7B
GPQA Diamond41%30.6%
MMLU80.3%70.6%
MMLU-Pro—33.5%
GPQA (HELM)—29.6%
LMArena Expert—1088
ARC (AI2) Challenge—87.3%
OpenBookQA—85.8%
TriviaQA—82.2%

Multimodal Not comparable

Llama 3.2 90B: 25.4 (#124), Mixtral 8x7B: —

Multimodal benchmarks
BenchmarkLlama 3.2 90BMixtral 8x7B
LMArena Vision1000—
GeoBench52%—

Multilingual Not comparable

Llama 3.2 90B: —, Mixtral 8x7B: 29.6 (#266)

Multilingual benchmarks
BenchmarkLlama 3.2 90BMixtral 8x7B
LMArena Non-English—1077
LMArena Chinese—1055
LMArena French—1166
LMArena German—1114
LMArena Japanese—931
LMArena Korean—968
LMArena Russian—1090
LMArena Spanish—1111

Instruction Following Not comparable

Llama 3.2 90B: —, Mixtral 8x7B: 51.0 (#297)

Instruction Following benchmarks
BenchmarkLlama 3.2 90BMixtral 8x7B
IFEval—57.5%
LMArena Instruction Following—1109

Long Context Not comparable

Llama 3.2 90B: —, Mixtral 8x7B: 33.4 (#260)

Long Context benchmarks
BenchmarkLlama 3.2 90BMixtral 8x7B
LMArena Longer Query—1103

Writing & Preference Not comparable

Llama 3.2 90B: —, Mixtral 8x7B: 34.2 (#270)

Writing & Preference benchmarks
BenchmarkLlama 3.2 90BMixtral 8x7B
LMArena Text—1132
LMArena Creative Writing—1109
WildBench—67.3%
LMArena Multi-Turn—1115

Frequently asked questions

Is Llama 3.2 90B better than Mixtral 8x7B?

Llama 3.2 90B and Mixtral 8x7B score almost the same on the Noometry Index (27.5 vs 27.1), so choose on price, context window or the category you care about most.

How many benchmarks do Llama 3.2 90B and Mixtral 8x7B share?

4 benchmarks have published results for both models. Llama 3.2 90B has 9 scored results on Noometry and Mixtral 8x7B has 38.

Related comparisons

Go deeper