Noometry

Model comparison

Deepseek Coder v2 vs Mistral Medium 3.5

Mistral Medium 3.5 is the stronger model overall, scoring 40.2 to 35.9 on the Noometry Index.

Last verified . 16 shared benchmarks.

Deepseek Coder v2 DeepSeek

35.9

Rank #220 Confirmed

Mistral Medium 3.5 Mistral AI

40.2

Rank #152 Confirmed

Summary

  • They share 16 benchmarks with published results for both. Deepseek Coder v2 scores higher in 2 categories and Mistral Medium 3.5 in 6 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where Mistral Medium 3.5 leads 58.5 to 38.2.

Side by side

Deepseek Coder v2Mistral Medium 3.5
ProviderDeepSeekMistral AI
Noometry Index35.940.2
Released2024-06-17—
WeightsOpenOpen
Context window—262K
Max output—210K
Input $ / M tokens—$1.50
Output $ / M tokens—$7.50
Results tracked2422

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Deepseek Coder v2 leads

Deepseek Coder v2: 38.1 (#183), Mistral Medium 3.5: 36.0 (#213)

BenchmarkDeepseek Coder v2Mistral Medium 3.5
LMArena Coding12511461
LMArena WebDev—1264
BigCodeBench Instruct48.2%—
BigCodeBench Complete59.7%—
HumanEval+82.3%—
MBPP+75.1%—

Reasoning Deepseek Coder v2 leads

Deepseek Coder v2: 23.6 (#176), Mistral Medium 3.5: 17.3 (#295)

BenchmarkDeepseek Coder v2Mistral Medium 3.5
LMArena Hard Prompts12071436
Kagi LLM Benchmark—41.4%
NYT Connections (extended)—12.9%
Epoch Capabilities Index—141.35
WinoGrande83.7%—

Math Mistral Medium 3.5 leads

Deepseek Coder v2: 34.9 (#190), Mistral Medium 3.5: 39.1 (#113)

BenchmarkDeepseek Coder v2Mistral Medium 3.5
LMArena Math12411431
GSM8K94.5%—

Knowledge Mistral Medium 3.5 leads

Deepseek Coder v2: 32.3 (#212), Mistral Medium 3.5: 40.0 (#126)

BenchmarkDeepseek Coder v2Mistral Medium 3.5
LMArena Expert11811432
ARC (AI2) Challenge64.3%—

Multimodal Not comparable

Deepseek Coder v2: —, Mistral Medium 3.5: 38.3 (#65)

BenchmarkDeepseek Coder v2Mistral Medium 3.5
LMArena Vision—1223

Multilingual Mistral Medium 3.5 leads

Deepseek Coder v2: 36.3 (#240), Mistral Medium 3.5: 51.9 (#100)

BenchmarkDeepseek Coder v2Mistral Medium 3.5
LMArena Non-English11821404
LMArena Chinese12011442
LMArena French11851448
LMArena German11641451
LMArena Korean11041385
LMArena Russian11881395
LMArena Spanish11531409
LMArena Japanese1126—

Instruction Following Mistral Medium 3.5 leads

Deepseek Coder v2: 61.7 (#242), Mistral Medium 3.5: 74.6 (#90)

BenchmarkDeepseek Coder v2Mistral Medium 3.5
LMArena Instruction Following11801415

Long Context Mistral Medium 3.5 leads

Deepseek Coder v2: 37.0 (#224), Mistral Medium 3.5: 43.2 (#103)

BenchmarkDeepseek Coder v2Mistral Medium 3.5
LMArena Longer Query12191415

Writing & Preference Mistral Medium 3.5 leads

Deepseek Coder v2: 38.2 (#253), Mistral Medium 3.5: 58.5 (#117)

BenchmarkDeepseek Coder v2Mistral Medium 3.5
LMArena Text11911421
LMArena Creative Writing11201374
LMArena Multi-Turn11771423
EQ-Bench 4—993

Frequently asked questions

Is Deepseek Coder v2 better than Mistral Medium 3.5?

Mistral Medium 3.5 is the stronger model overall, scoring 40.2 to 35.9 on the Noometry Index.

Is Deepseek Coder v2 or Mistral Medium 3.5 better for coding?

Deepseek Coder v2 scores higher on coding benchmarks: 38.1 versus 36.0 in the Noometry coding category.

How many benchmarks do Deepseek Coder v2 and Mistral Medium 3.5 share?

16 benchmarks have published results for both models. Deepseek Coder v2 has 24 scored results on Noometry and Mistral Medium 3.5 has 22.

Related comparisons

Go deeper