Noometry

Model comparison

Mixtral 8x22B vs Nemotron 4 340b Instruct

Nemotron 4 340b Instruct is the stronger model overall, scoring 35.3 to 26.7 on the Noometry Index.

Last verified . 17 shared benchmarks.

Mixtral 8x22B Mistral AI

26.7

Rank #330 Confirmed

Nemotron 4 340b Instruct NVIDIA

35.3

Rank #218 Confirmed

Summary

  • They share 17 benchmarks with published results for both. Mixtral 8x22B scores higher in 0 categories and Nemotron 4 340b Instruct in 8 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in knowledge, where Nemotron 4 340b Instruct leads 31.4 to 15.1.

Side by side

Mixtral 8x22BNemotron 4 340b Instruct
ProviderMistral AINVIDIA
Noometry Index26.735.3
Released2024-04-172024-06-14
WeightsOpenOpen
Context window64K—
Max output64K—
Input $ / M tokens$2—
Output $ / M tokens$6—
Results tracked3417

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Nemotron 4 340b Instruct leads

Mixtral 8x22B: 23.8 (#324), Nemotron 4 340b Instruct: 34.6 (#223)

BenchmarkMixtral 8x22BNemotron 4 340b Instruct
LMArena Coding11661209
WeirdML3.2%—
BigCodeBench Instruct40.6%—
BigCodeBench Complete50.2%—
HumanEval+72%—
MBPP+64.3%—

Agentic & Tool Use Not comparable

Mixtral 8x22B: 22.8 (#119), Nemotron 4 340b Instruct: —

BenchmarkMixtral 8x22BNemotron 4 340b Instruct
Cybench7.5%—

Reasoning Nemotron 4 340b Instruct leads

Mixtral 8x22B: 19.1 (#246), Nemotron 4 340b Instruct: 22.4 (#180)

BenchmarkMixtral 8x22BNemotron 4 340b Instruct
LMArena Hard Prompts11501202
DTBench55.1%—
Epoch Capabilities Index122—
ForecastBench56.3—

Math Nemotron 4 340b Instruct leads

Mixtral 8x22B: 21.2 (#271), Nemotron 4 340b Instruct: 31.7 (#193)

BenchmarkMixtral 8x22BNemotron 4 340b Instruct
LMArena Math11841216
Omni-MATH16.3%—
MATH Level 524.2%—

Knowledge Nemotron 4 340b Instruct leads

Mixtral 8x22B: 15.1 (#288), Nemotron 4 340b Instruct: 31.4 (#212)

BenchmarkMixtral 8x22BNemotron 4 340b Instruct
LMArena Expert11131171
GPQA Diamond34.1%—
MMLU-Pro46%—
GPQA (HELM)33.4%—
MMLU77.8%—

Multilingual Nemotron 4 340b Instruct leads

Mixtral 8x22B: 32.9 (#250), Nemotron 4 340b Instruct: 38.0 (#228)

BenchmarkMixtral 8x22BNemotron 4 340b Instruct
LMArena Non-English11281206
LMArena Chinese11161214
LMArena French11651196
LMArena German11411192
LMArena Japanese10371128
LMArena Korean10561148
LMArena Russian11581219
LMArena Spanish11501195

Instruction Following Nemotron 4 340b Instruct leads

Mixtral 8x22B: 57.9 (#261), Nemotron 4 340b Instruct: 63.3 (#227)

BenchmarkMixtral 8x22BNemotron 4 340b Instruct
LMArena Instruction Following11471203
IFEval72.4%—

Long Context Nemotron 4 340b Instruct leads

Mixtral 8x22B: 34.6 (#242), Nemotron 4 340b Instruct: 37.1 (#216)

BenchmarkMixtral 8x22BNemotron 4 340b Instruct
LMArena Longer Query11441225

Writing & Preference Nemotron 4 340b Instruct leads

Mixtral 8x22B: 37.1 (#257), Nemotron 4 340b Instruct: 42.9 (#229)

BenchmarkMixtral 8x22BNemotron 4 340b Instruct
LMArena Text11621225
LMArena Creative Writing11411203
LMArena Multi-Turn11301214
WildBench71.1%—

Frequently asked questions

Is Mixtral 8x22B better than Nemotron 4 340b Instruct?

Nemotron 4 340b Instruct is the stronger model overall, scoring 35.3 to 26.7 on the Noometry Index.

Is Mixtral 8x22B or Nemotron 4 340b Instruct better for coding?

Nemotron 4 340b Instruct scores higher on coding benchmarks: 34.6 versus 23.8 in the Noometry coding category.

How many benchmarks do Mixtral 8x22B and Nemotron 4 340b Instruct share?

17 benchmarks have published results for both models. Mixtral 8x22B has 34 scored results on Noometry and Nemotron 4 340b Instruct has 17.

Related comparisons

Go deeper