Noometry

Model comparison

DBRX vs Falcon-180B

Falcon-180B is the stronger model overall, scoring 32.2 to 29.4 on the Noometry Index.

Last verified . 6 shared benchmarks.

DBRX Databricks

29.4

Rank #311 Confirmed

Falcon-180B Technology Innovation Institute

32.2

Rank #260 Confirmed

Summary

  • They share 6 benchmarks with published results for both. DBRX scores higher in 4 categories and Falcon-180B in 0 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in writing & preference, where DBRX leads 33.6 to 29.1.

Side by side

DBRXFalcon-180B
ProviderDatabricksTechnology Innovation Institute
Noometry Index29.432.2
Released2024-03-272023-09-06
WeightsOpenOpen
Context window——
Max output——
Input $ / M tokens——
Output $ / M tokens——
Results tracked2116

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Not comparable

DBRX: 32.9 (#266), Falcon-180B: —

BenchmarkDBRXFalcon-180B
LMArena Coding1132—
HumanEval+70.1%—
MBPP+55.8%—

Reasoning DBRX leads

DBRX: 21.4 (#222), Falcon-180B: 19.1 (#269)

BenchmarkDBRXFalcon-180B
LMArena Hard Prompts11131007
Epoch Capabilities Index—112.13
HellaSwag—89%
LAMBADA—79.8%
PIQA—84.9%
WinoGrande—87.1%

Math Not comparable

DBRX: 24.3 (#269), Falcon-180B: —

BenchmarkDBRXFalcon-180B
LMArena Math1145—
MATH Level 511.7%—
GSM8K—54.4%

Knowledge Not comparable

DBRX: 14.9 (#294), Falcon-180B: —

BenchmarkDBRXFalcon-180B
GPQA Diamond32.9%—
LMArena Expert1076—
ARC (AI2) Challenge—67.8%
BoolQ—89%
MMLU—70.6%
OpenBookQA—64.2%

Multilingual DBRX leads

DBRX: 29.3 (#268), Falcon-180B: 25.2 (#286)

BenchmarkDBRXFalcon-180B
LMArena Non-English10711000
LMArena Chinese1068—
LMArena French1096—
LMArena German1057—
LMArena Japanese990—
LMArena Korean993—
LMArena Russian1078—
LMArena Spanish1064—

Instruction Following DBRX leads

DBRX: 57.5 (#270), Falcon-180B: 53.4 (#286)

BenchmarkDBRXFalcon-180B
LMArena Instruction Following11121047

Long Context Not comparable

DBRX: 33.7 (#258), Falcon-180B: —

BenchmarkDBRXFalcon-180B
LMArena Longer Query1112—

Writing & Preference DBRX leads

DBRX: 33.6 (#275), Falcon-180B: 29.1 (#295)

BenchmarkDBRXFalcon-180B
LMArena Text11191054
LMArena Creative Writing11041089
LMArena Multi-Turn11111013

Frequently asked questions

Is DBRX better than Falcon-180B?

Falcon-180B is the stronger model overall, scoring 32.2 to 29.4 on the Noometry Index.

How many benchmarks do DBRX and Falcon-180B share?

6 benchmarks have published results for both models. DBRX has 21 scored results on Noometry and Falcon-180B has 16.

Related comparisons

Go deeper