Noometry

Model comparison

Ministral 3B vs Mixtral 8x7B

Ministral 3B and Mixtral 8x7B score almost the same on the Noometry Index (26.2 vs 27.1), so choose on price, context window or the category you care about most.

Last verified . 4 shared benchmarks.

Ministral 3B Mistral AI

26.2

Rank #338 Confirmed

Mixtral 8x7B Mistral AI

27.1

Rank #334 Confirmed

Summary

  • They share 4 benchmarks with published results for both. Ministral 3B scores higher in 2 categories and Mixtral 8x7B in 1 category; one gap is clear of the uncertainty.
  • The widest gap is in math, where Ministral 3B leads 26.6 to 18.8.
  • The biggest single-benchmark swing is GPQA Diamond: 25.3% for Ministral 3B and 30.6% for Mixtral 8x7B.
  • Ministral 3B is cheaper at $0.10 / $0.10 per million input/output tokens, against $0.70 / $0.70 for Mixtral 8x7B.
  • Ministral 3B accepts more context: 131K tokens versus 32K.

Side by side

Ministral 3BMixtral 8x7B
ProviderMistral AIMistral AI
Noometry Index26.227.1
Released2024-10-012023-12-11
WeightsOpenOpen
Context window131K32K
Max output262K32K
Input $ / M tokens$0.10$0.70
Output $ / M tokens$0.10$0.70
Results tracked638

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

Ministral 3B: —, Mixtral 8x7B: 32.8 (#269)

BenchmarkMinistral 3BMixtral 8x7B
LMArena Coding—1126
HumanEval+—39.6%
MBPP+—49.7%

Reasoning Too close to call

Ministral 3B: 18.4 (#282), Mixtral 8x7B: 18.2 (#285)

BenchmarkMinistral 3BMixtral 8x7B
DTBench51.7%49.6%
Epoch Capabilities Index118.1118.47
LMArena Hard Prompts—1115
LMCA5.5%—
Adversarial NLI—55.2%
ForecastBench—56.3
HellaSwag—86.7%
PIQA—83.6%
WinoGrande—77.2%

Math Ministral 3B leads

Ministral 3B: 26.6 (#258), Mixtral 8x7B: 18.8 (#289)

BenchmarkMinistral 3BMixtral 8x7B
MATH Level 514.4%10%
Omni-MATH—10.5%
LMArena Math—1147
GSM8K—74.4%

Knowledge Too close to call

Ministral 3B: 10.4 (#302), Mixtral 8x7B: 11.0 (#301)

BenchmarkMinistral 3BMixtral 8x7B
GPQA Diamond25.3%30.6%
MMLU-Pro—33.5%
Vectara Hallucination Rate7.3%—
GPQA (HELM)—29.6%
LMArena Expert—1088
ARC (AI2) Challenge—87.3%
MMLU—70.6%
OpenBookQA—85.8%
TriviaQA—82.2%

Multilingual Not comparable

Ministral 3B: —, Mixtral 8x7B: 29.6 (#266)

BenchmarkMinistral 3BMixtral 8x7B
LMArena Non-English—1077
LMArena Chinese—1055
LMArena French—1166
LMArena German—1114
LMArena Japanese—931
LMArena Korean—968
LMArena Russian—1090
LMArena Spanish—1111

Instruction Following Not comparable

Ministral 3B: —, Mixtral 8x7B: 51.0 (#297)

BenchmarkMinistral 3BMixtral 8x7B
IFEval—57.5%
LMArena Instruction Following—1109

Long Context Not comparable

Ministral 3B: —, Mixtral 8x7B: 33.4 (#260)

BenchmarkMinistral 3BMixtral 8x7B
LMArena Longer Query—1103

Writing & Preference Not comparable

Ministral 3B: —, Mixtral 8x7B: 34.2 (#270)

BenchmarkMinistral 3BMixtral 8x7B
LMArena Text—1132
LMArena Creative Writing—1109
WildBench—67.3%
LMArena Multi-Turn—1115

Frequently asked questions

Is Ministral 3B better than Mixtral 8x7B?

Ministral 3B and Mixtral 8x7B score almost the same on the Noometry Index (26.2 vs 27.1), so choose on price, context window or the category you care about most.

Which is cheaper, Ministral 3B or Mixtral 8x7B?

Ministral 3B is cheaper. It lists at $0.10 per million input tokens and $0.10 per million output tokens; Mixtral 8x7B lists at $0.70 and $0.70.

Which has the bigger context window?

Ministral 3B does, with 131K tokens against 32K.

How many benchmarks do Ministral 3B and Mixtral 8x7B share?

4 benchmarks have published results for both models. Ministral 3B has 6 scored results on Noometry and Mixtral 8x7B has 38.

Related comparisons

Go deeper