Noometry

Model comparison

DBRX vs GPT-4o mini

DBRX is the stronger model overall, scoring 29.4 to 25.5 on the Noometry Index.

Last verified . 21 shared benchmarks.

DBRX Databricks

29.4

Rank #311 Confirmed

GPT-4o mini OpenAI

25.5

Rank #343 Confirmed

Summary

  • They share 21 benchmarks with published results for both. DBRX scores higher in 3 categories and GPT-4o mini in 5 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where DBRX leads 24.3 to 10.4.
  • The biggest single-benchmark swing is MATH Level 5: 11.7% for DBRX and 52.6% for GPT-4o mini.
  • DBRX has downloadable open weights; the other is API-only.

Side by side

DBRXGPT-4o mini
ProviderDatabricksOpenAI
Noometry Index29.425.5
Released2024-03-272024-07-18
WeightsOpenProprietary
Context window—128K
Max output—16K
Input $ / M tokens—$0.15
Output $ / M tokens—$0.60
Results tracked2160

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding DBRX leads

DBRX: 32.9 (#266), GPT-4o mini: 22.0 (#335)

BenchmarkDBRXGPT-4o mini
LMArena Coding11321290
HumanEval+70.1%83.5%
MBPP+55.8%72.2%
Aider Polyglot—3.6%
WeirdML—11.8%
BigCodeBench Instruct—46.1%
LiveBench Coding—43.1%
BigCodeBench Complete—57.4%

Agentic & Tool Use Not comparable

DBRX: —, GPT-4o mini: 27.5 (#101)

BenchmarkDBRXGPT-4o mini
BALROG—17.4%

Reasoning DBRX leads

DBRX: 21.4 (#222), GPT-4o mini: 8.7 (#347)

BenchmarkDBRXGPT-4o mini
LMArena Hard Prompts11131267
ARC-AGI-2—0%
SimpleBench—10.7%
Kagi LLM Benchmark—28.8%
Chess Puzzles—0%
LiveBench Reasoning—32.8%
Mystery Game Puzzles—12%
DTBench—54.4%
LiveBench Data Analysis—50%
LMCA—10.4%
Epoch Capabilities Index—126.56
LiveBench—41.3%
PIQA—88.7%

Math DBRX leads

DBRX: 24.3 (#269), GPT-4o mini: 10.4 (#314)

BenchmarkDBRXGPT-4o mini
LMArena Math11451267
MATH Level 511.7%52.6%
FrontierMath (Tiers 1-3)—0.7%
OTIS Mock AIME 2024-2025—6.9%
Omni-MATH—28%
LiveBench Math—36.3%
GSM8K—91.3%

Knowledge GPT-4o mini leads

DBRX: 14.9 (#294), GPT-4o mini: 17.7 (#284)

BenchmarkDBRXGPT-4o mini
GPQA Diamond32.9%37.7%
LMArena Expert10761235
SimpleQA Verified—8.3%
MMLU-Pro—60.3%
Confabulations—37.2%
GPQA (HELM)—36.8%
BoolQ—88.7%
MMLU—81.8%

Multimodal Not comparable

DBRX: —, GPT-4o mini: 25.9 (#122)

BenchmarkDBRXGPT-4o mini
LMArena Vision—1066
Video-MME—64.8%
GeoBench—64%
VPCT—34%

Multilingual GPT-4o mini leads

DBRX: 29.3 (#268), GPT-4o mini: 42.0 (#199)

BenchmarkDBRXGPT-4o mini
LMArena Non-English10711266
LMArena Chinese10681265
LMArena French10961297
LMArena German10571272
LMArena Japanese9901216
LMArena Korean9931195
LMArena Russian10781275
LMArena Spanish10641276

Instruction Following GPT-4o mini leads

DBRX: 57.5 (#270), GPT-4o mini: 61.9 (#239)

BenchmarkDBRXGPT-4o mini
LMArena Instruction Following11121258
LiveBench Instruction Following—56.8%
IFEval—78.2%

Long Context GPT-4o mini leads

DBRX: 33.7 (#258), GPT-4o mini: 39.1 (#186)

BenchmarkDBRXGPT-4o mini
LMArena Longer Query11121289

Writing & Preference GPT-4o mini leads

DBRX: 33.6 (#275), GPT-4o mini: 39.5 (#248)

BenchmarkDBRXGPT-4o mini
LMArena Text11191286
LMArena Creative Writing11041268
LMArena Multi-Turn11111285
Short-Story Creative Writing—67.2%
EQ-Bench Creative Writing—873
WildBench—79.1%
LiveBench Language—28.6%

Frequently asked questions

Is DBRX better than GPT-4o mini?

DBRX is the stronger model overall, scoring 29.4 to 25.5 on the Noometry Index.

Is DBRX or GPT-4o mini better for coding?

DBRX scores higher on coding benchmarks: 32.9 versus 22.0 in the Noometry coding category.

How many benchmarks do DBRX and GPT-4o mini share?

21 benchmarks have published results for both models. DBRX has 21 scored results on Noometry and GPT-4o mini has 60.

Related comparisons

Go deeper