Noometry

Model comparison

Command R vs Mixtral 8x7B

Command R is the stronger model overall, scoring 31.0 to 26.8 on the Noometry Index.

Last verified . 19 shared benchmarks.

Command R Cohere

31.0

Rank #267 Confirmed

Mixtral 8x7B Mistral AI

26.8

Rank #328 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Command R scores higher in 6 categories and Mixtral 8x7B in 2 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Command R leads 30.5 to 11.0.
  • Command R is cheaper at $0.15 / $0.60 per million input/output tokens, against $0.70 / $0.70 for Mixtral 8x7B.
  • Command R accepts more context: 128K tokens versus 32K.

Side by side

Command RMixtral 8x7B
ProviderCohereMistral AI
Noometry Index31.026.8
Released2024-08-302023-12-11
WeightsOpenOpen
Context window128K32K
Max output4K32K
Input $ / M tokens$0.15$0.70
Output $ / M tokens$0.60$0.70
Results tracked2938

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Mixtral 8x7B leads

Command R: 28.8 (#301), Mixtral 8x7B: 32.2 (#265)

BenchmarkCommand RMixtral 8x7B
LMArena Coding11701127
BigCodeBench Instruct37.1%—
LiveBench Coding17.9%—
BigCodeBench Complete45.2%—
HumanEval+—39.6%
MBPP+—49.7%

Reasoning Mixtral 8x7B leads

Command R: 13.3 (#329), Mixtral 8x7B: 17.4 (#283)

BenchmarkCommand RMixtral 8x7B
LMArena Hard Prompts11641115
DTBench46.4%49.6%
LiveBench Reasoning21.9%—
LiveBench Data Analysis33.3%—
LMCA9.2%—
Adversarial NLI—55.2%
Epoch Capabilities Index—118.38
ForecastBench—56.3
HellaSwag—86.7%
LiveBench27.5%—
PIQA—83.6%
WinoGrande—77.2%

Math Command R leads

Command R: 25.8 (#244), Mixtral 8x7B: 17.2 (#284)

BenchmarkCommand RMixtral 8x7B
LMArena Math11551147
Omni-MATH—10.5%
LiveBench Math19.4%—
MATH Level 5—10%
GSM8K—74.4%

Knowledge Command R leads

Command R: 30.5 (#219), Mixtral 8x7B: 11.0 (#296)

BenchmarkCommand RMixtral 8x7B
LMArena Expert11381088
MMLU65.2%70.6%
GPQA Diamond—30.6%
MMLU-Pro—33.5%
GPQA (HELM)—29.6%
ARC (AI2) Challenge—87.3%
OpenBookQA—85.8%
TriviaQA—82.2%

Multilingual Command R leads

Command R: 35.9 (#240), Mixtral 8x7B: 29.8 (#261)

BenchmarkCommand RMixtral 8x7B
LMArena Non-English11741077
LMArena Chinese11821055
LMArena French11611165
LMArena German11761114
LMArena Japanese1143931
LMArena Korean1162968
LMArena Russian11741090
LMArena Spanish11501110

Instruction Following Command R leads

Command R: 58.3 (#256), Mixtral 8x7B: 51.2 (#292)

BenchmarkCommand RMixtral 8x7B
LMArena Instruction Following11671110
LiveBench Instruction Following55.6%—
IFEval—57.5%

Long Context Command R leads

Command R: 36.3 (#226), Mixtral 8x7B: 33.4 (#255)

BenchmarkCommand RMixtral 8x7B
LMArena Longer Query11981103

Writing & Preference Command R leads

Command R: 38.4 (#248), Mixtral 8x7B: 34.4 (#265)

BenchmarkCommand RMixtral 8x7B
LMArena Text11871132
LMArena Creative Writing11701109
LMArena Multi-Turn11631115
WildBench—67.3%
LiveBench Language16.7%—

Frequently asked questions

Is Command R better than Mixtral 8x7B?

Command R is the stronger model overall, scoring 31.0 to 26.8 on the Noometry Index.

Which is cheaper, Command R or Mixtral 8x7B?

Command R is cheaper. It lists at $0.15 per million input tokens and $0.60 per million output tokens; Mixtral 8x7B lists at $0.70 and $0.70.

Is Command R or Mixtral 8x7B better for coding?

Mixtral 8x7B scores higher on coding benchmarks: 32.2 versus 28.8 in the Noometry coding category.

Which has the bigger context window?

Command R does, with 128K tokens against 32K.

How many benchmarks do Command R and Mixtral 8x7B share?

19 benchmarks have published results for both models. Command R has 29 scored results on Noometry and Mixtral 8x7B has 38.

Related comparisons

Go deeper