Model comparison
Mistral 7B vs Mixtral 8x22B
Mixtral 8x22B is the stronger model overall, scoring 27.1 to 23.0 on the Noometry Index. Mistral 7B costs 12× less per token, which makes it the better buy when Mixtral 8x22B's lead doesn't matter for your workload.
Last verified . 25 shared benchmarks.
Mistral AISummary
- They share 25 benchmarks with published results for both. Mistral 7B scores higher in 1 category and Mixtral 8x22B in 7 categories; 8 gaps are clear of the uncertainty.
- The widest gap is in math, where Mixtral 8x22B leads 22.9 to 8.1.
- The biggest single-benchmark swing is BigCodeBench Complete: 27.3% for Mistral 7B and 50.2% for Mixtral 8x22B.
- Mistral 7B is cheaper at $0.25 / $0.25 per million input/output tokens, against $2 / $6 for Mixtral 8x22B.
- Mixtral 8x22B accepts more context: 64K tokens versus 8K.
Side by side
| Mistral 7B | Mixtral 8x22B | |
|---|---|---|
| Provider | Mistral AI | Mistral AI |
| Noometry Index | 23.0 | 27.1 |
| Released | 2023-09-27 | 2024-04-17 |
| Weights | Open | Open |
| Context window | 8K | 64K |
| Max output | 8K | 64K |
| Input $ / M tokens | $0.25 | $2 |
| Output $ / M tokens | $0.25 | $6 |
| Results tracked | 37 | 34 |
Sponsored placements are available on pages like this one. Advertise on Noometry
Category by category
Coding Mistral 7B leads
Mistral 7B: 26.4 (#326), Mixtral 8x22B: 24.2 (#329)
| Benchmark | Mistral 7B | Mixtral 8x22B |
|---|---|---|
| BigCodeBench Instruct | 19.5% | 40.6% |
| LMArena Coding | 1082 | 1166 |
| BigCodeBench Complete | 27.3% | 50.2% |
| HumanEval+ | 36% | 72% |
| MBPP+ | 42.1% | 64.3% |
| WeirdML | — | 3.2% |
Agentic & Tool Use Not comparable
Mistral 7B: —, Mixtral 8x22B: 23.1 (#127)
| Benchmark | Mistral 7B | Mixtral 8x22B |
|---|---|---|
| Cybench | — | 7.5% |
Reasoning Mixtral 8x22B leads
Mistral 7B: 13.1 (#336), Mixtral 8x22B: 19.9 (#248)
| Benchmark | Mistral 7B | Mixtral 8x22B |
|---|---|---|
| LMArena Hard Prompts | 1067 | 1150 |
| DTBench | 42.5% | 55.1% |
| Epoch Capabilities Index | 112.21 | 122.03 |
| Chess Puzzles | 0% | — |
| Adversarial NLI | 47.1% | — |
| BIG-Bench Hard | 56.1% | — |
| ForecastBench | — | 56.3 |
| HellaSwag | 81% | — |
| PIQA | 83% | — |
| WinoGrande | 75.3% | — |
Math Mixtral 8x22B leads
Mistral 7B: 8.1 (#325), Mixtral 8x22B: 22.9 (#275)
| Benchmark | Mistral 7B | Mixtral 8x22B |
|---|---|---|
| LMArena Math | 1085 | 1184 |
| MATH Level 5 | 3.7% | 24.2% |
| OTIS Mock AIME 2024-2025 | 0.3% | — |
| Omni-MATH | — | 16.3% |
| GSM8K | 54.4% | — |
Knowledge Mixtral 8x22B leads
Mistral 7B: 7.4 (#311), Mixtral 8x22B: 15.1 (#293)
| Benchmark | Mistral 7B | Mixtral 8x22B |
|---|---|---|
| GPQA Diamond | 15.2% | 34.1% |
| LMArena Expert | 1036 | 1113 |
| MMLU | 62.5% | 77.8% |
| MMLU-Pro | — | 46% |
| GPQA (HELM) | — | 33.4% |
| ARC (AI2) Challenge | 78.6% | — |
| BoolQ | 87.4% | — |
| OpenBookQA | 79.8% | — |
| TriviaQA | 75.2% | — |
Multilingual Mixtral 8x22B leads
Mistral 7B: 25.8 (#283), Mixtral 8x22B: 32.8 (#255)
| Benchmark | Mistral 7B | Mixtral 8x22B |
|---|---|---|
| LMArena Non-English | 1012 | 1128 |
| LMArena Chinese | 1009 | 1116 |
| LMArena French | 1037 | 1166 |
| LMArena German | 987 | 1141 |
| LMArena Japanese | 878 | 1037 |
| LMArena Russian | 1018 | 1158 |
| LMArena Spanish | 1026 | 1151 |
| LMArena Korean | — | 1057 |
Instruction Following Mixtral 8x22B leads
Mistral 7B: 54.2 (#280), Mixtral 8x22B: 57.7 (#266)
| Benchmark | Mistral 7B | Mixtral 8x22B |
|---|---|---|
| LMArena Instruction Following | 1060 | 1147 |
| IFEval | — | 72.4% |
Long Context Mixtral 8x22B leads
Mistral 7B: 32.2 (#271), Mixtral 8x22B: 34.7 (#247)
| Benchmark | Mistral 7B | Mixtral 8x22B |
|---|---|---|
| LMArena Longer Query | 1060 | 1144 |
Writing & Preference Mixtral 8x22B leads
Mistral 7B: 30.7 (#286), Mixtral 8x22B: 36.9 (#262)
| Benchmark | Mistral 7B | Mixtral 8x22B |
|---|---|---|
| LMArena Text | 1090 | 1162 |
| LMArena Creative Writing | 1068 | 1141 |
| LMArena Multi-Turn | 1062 | 1130 |
| WildBench | — | 71.1% |
Frequently asked questions
Is Mistral 7B better than Mixtral 8x22B?
Mixtral 8x22B is the stronger model overall, scoring 27.1 to 23.0 on the Noometry Index. Mistral 7B costs 12× less per token, which makes it the better buy when Mixtral 8x22B's lead doesn't matter for your workload.
Which is cheaper, Mistral 7B or Mixtral 8x22B?
Mistral 7B is cheaper. It lists at $0.25 per million input tokens and $0.25 per million output tokens; Mixtral 8x22B lists at $2 and $6.
Is Mistral 7B or Mixtral 8x22B better for coding?
Mistral 7B scores higher on coding benchmarks: 26.4 versus 24.2 in the Noometry coding category.
Which has the bigger context window?
Mixtral 8x22B does, with 64K tokens against 8K.
How many benchmarks do Mistral 7B and Mixtral 8x22B share?
25 benchmarks have published results for both models. Mistral 7B has 37 scored results on Noometry and Mixtral 8x22B has 34.