Noometry

Model comparison

gpt-oss-20b vs Mistral 7B

gpt-oss-20b is the stronger model overall, scoring 32.1 to 22.9 on the Noometry Index.

Last verified . 20 shared benchmarks.

gpt-oss-20b OpenAI

32.1

Rank #249 Confirmed

Mistral 7B Mistral AI

22.9

Rank #346 Confirmed

Summary

  • They share 20 benchmarks with published results for both. gpt-oss-20b scores higher in 8 categories and Mistral 7B in 0 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where gpt-oss-20b leads 37.6 to 7.6.
  • The biggest single-benchmark swing is OTIS Mock AIME 2024-2025: 65.3% for gpt-oss-20b and 0.3% for Mistral 7B.
  • gpt-oss-20b is cheaper at $0.018 / $0.09 per million input/output tokens, against $0.25 / $0.25 for Mistral 7B.
  • gpt-oss-20b accepts more context: 131K tokens versus 8K.

Side by side

gpt-oss-20bMistral 7B
ProviderOpenAIMistral AI
Noometry Index32.122.9
Released2025-08-052023-09-27
WeightsOpenOpen
Context window131K8K
Max output16K8K
Input $ / M tokens$0.018$0.25
Output $ / M tokens$0.09$0.25
Results tracked3437

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding gpt-oss-20b leads

gpt-oss-20b: 37.1 (#185), Mistral 7B: 25.9 (#321)

Benchmarkgpt-oss-20bMistral 7B
LMArena Coding13071082
SciCode34.4%—
WeirdML40.9%—
BigCodeBench Instruct—19.5%
BigCodeBench Complete—27.3%
ALE-Bench566.05—
HumanEval+—36%
MBPP+—42.1%

Agentic & Tool Use Not comparable

gpt-oss-20b: 9.2 (#146), Mistral 7B: —

Benchmarkgpt-oss-20bMistral 7B
Terminal-Bench3.4%—

Reasoning gpt-oss-20b leads

gpt-oss-20b: 18.8 (#251), Mistral 7B: 12.6 (#331)

Benchmarkgpt-oss-20bMistral 7B
Chess Puzzles4%0%
LMArena Hard Prompts12741067
DTBench68%42.5%
Epoch Capabilities Index137.81111.97
Kagi LLM Benchmark53.2%—
CritPt1.4%—
LMCA14.5%—
Adversarial NLI—47.1%
BIG-Bench Hard—56.1%
HellaSwag—81%
PIQA—83%
WinoGrande—75.3%

Math gpt-oss-20b leads

gpt-oss-20b: 37.6 (#93), Mistral 7B: 7.6 (#319)

Benchmarkgpt-oss-20bMistral 7B
OTIS Mock AIME 2024-202565.3%0.3%
LMArena Math13171085
Omni-MATH56.5%—
MATH Level 5—3.7%
GSM8K—54.4%

Knowledge gpt-oss-20b leads

gpt-oss-20b: 34.6 (#186), Mistral 7B: 7.3 (#306)

Benchmarkgpt-oss-20bMistral 7B
GPQA Diamond60.8%15.2%
LMArena Expert12581036
MMLU-Pro74%—
GPQA (HELM)59.4%—
ARC (AI2) Challenge—78.6%
BoolQ—87.4%
MMLU—62.5%
OpenBookQA—79.8%
TriviaQA—75.2%

Multilingual gpt-oss-20b leads

gpt-oss-20b: 42.4 (#192), Mistral 7B: 26.0 (#278)

Benchmarkgpt-oss-20bMistral 7B
LMArena Non-English12691012
LMArena Chinese13111009
LMArena German1249987
LMArena Japanese1245878
LMArena Russian12791018
LMArena Spanish12671026
LMArena French—1036
LMArena Korean1239—

Instruction Following gpt-oss-20b leads

gpt-oss-20b: 62.0 (#235), Mistral 7B: 54.5 (#275)

Benchmarkgpt-oss-20bMistral 7B
LMArena Instruction Following12361060
IFEval73.2%—

Long Context gpt-oss-20b leads

gpt-oss-20b: 37.9 (#203), Mistral 7B: 32.2 (#266)

Benchmarkgpt-oss-20bMistral 7B
LMArena Longer Query12511060

Writing & Preference gpt-oss-20b leads

gpt-oss-20b: 35.8 (#260), Mistral 7B: 30.9 (#281)

Benchmarkgpt-oss-20bMistral 7B
LMArena Text12881090
LMArena Creative Writing12011068
LMArena Multi-Turn12681062
EQ-Bench Creative Writing666—
WildBench73.7%—

Frequently asked questions

Is gpt-oss-20b better than Mistral 7B?

gpt-oss-20b is the stronger model overall, scoring 32.1 to 22.9 on the Noometry Index.

Which is cheaper, gpt-oss-20b or Mistral 7B?

gpt-oss-20b is cheaper. It lists at $0.018 per million input tokens and $0.09 per million output tokens; Mistral 7B lists at $0.25 and $0.25.

Is gpt-oss-20b or Mistral 7B better for coding?

gpt-oss-20b scores higher on coding benchmarks: 37.1 versus 25.9 in the Noometry coding category.

Which has the bigger context window?

gpt-oss-20b does, with 131K tokens against 8K.

How many benchmarks do gpt-oss-20b and Mistral 7B share?

20 benchmarks have published results for both models. gpt-oss-20b has 34 scored results on Noometry and Mistral 7B has 37.

Related comparisons

Go deeper