Noometry

Model comparison

Command R+ vs GPT-4

Command R+ is the stronger model overall, scoring 32.4 to 29.1 on the Noometry Index.

Last verified . 24 shared benchmarks.

Command R+ Cohere

32.4

Rank #257 Confirmed

GPT-4 OpenAI

29.1

Rank #316 Confirmed

Summary

  • They share 24 benchmarks with published results for both. Command R+ scores higher in 3 categories and GPT-4 in 5 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Command R+ leads 28.9 to 10.8.
  • The biggest single-benchmark swing is BigCodeBench Complete: 41.9% for Command R+ and 57.2% for GPT-4.
  • Command R+ is cheaper at $2.50 / $10 per million input/output tokens, against $30 / $60 for GPT-4.
  • Command R+ accepts more context: 128K tokens versus 8K.
  • Command R+ has downloadable open weights; the other is API-only.

Side by side

Command R+GPT-4
ProviderCohereOpenAI
Noometry Index32.429.1
Released2024-08-302023-03-14
WeightsOpenProprietary
Context window128K8K
Max output4K8K
Input $ / M tokens$2.50$30
Output $ / M tokens$10$60
Results tracked3438

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding GPT-4 leads

Command R+: 29.1 (#309), GPT-4: 31.6 (#283)

BenchmarkCommand R+GPT-4
BigCodeBench Instruct33.8%46%
LMArena Coding11871254
BigCodeBench Complete41.9%57.2%
HumanEval+56.7%79.3%
WeirdML—12.4%
LiveBench Coding19.1%—
MBPP+63.5%—

Agentic & Tool Use Not comparable

Command R+: —, GPT-4: —

BenchmarkCommand R+GPT-4
METR Time Horizons—36.1%

Reasoning GPT-4 leads

Command R+: 9.2 (#344), GPT-4: 17.8 (#289)

BenchmarkCommand R+GPT-4
LMArena Hard Prompts11861241
DTBench54.9%62.7%
LMCA5%17.1%
Epoch Capabilities Index119.34125.89
SimpleBench17.4%—
Chess Puzzles—4%
LiveBench Reasoning24.8%—
Mystery Game Puzzles—12%
LiveBench Data Analysis38.1%—
BIG-Bench Hard—75.1%
ForecastBench—57.8
HellaSwag—95.3%
LiveBench31.8%—
WinoGrande—87.5%

Math Command R+ leads

Command R+: 28.9 (#242), GPT-4: 10.8 (#309)

BenchmarkCommand R+GPT-4
LMArena Math11881269
OTIS Mock AIME 2024-2025—1.1%
LiveBench Math21.3%—
MATH Level 5—23%
GSM8K—92%

Knowledge Command R+ leads

Command R+: 36.4 (#169), GPT-4: 18.4 (#282)

BenchmarkCommand R+GPT-4
LMArena Expert11741211
MMLU69.4%86.4%
GPQA Diamond—35.7%
Vectara Hallucination Rate6.9%—
TriviaQA—84.8%

Multilingual GPT-4 leads

Command R+: 38.6 (#227), GPT-4: 40.6 (#215)

BenchmarkCommand R+GPT-4
LMArena Non-English12161246
LMArena Chinese12261242
LMArena French12091283
LMArena German12161251
LMArena Japanese11661209
LMArena Korean11381184
LMArena Russian12271251
LMArena Spanish11891261

Instruction Following GPT-4 leads

Command R+: 60.0 (#254), GPT-4: 65.3 (#222)

BenchmarkCommand R+GPT-4
LMArena Instruction Following11971241
LiveBench Instruction Following57.6%—

Long Context Too close to call

Command R+: 37.3 (#219), GPT-4: 37.7 (#212)

BenchmarkCommand R+GPT-4
LMArena Longer Query12301244

Writing & Preference Command R+ leads

Command R+: 43.5 (#228), GPT-4: 34.9 (#268)

BenchmarkCommand R+GPT-4
LMArena Text12291263
LMArena Creative Writing12351244
LMArena Multi-Turn12131257
EQ-Bench Creative Writing—752
LiveBench Language29.7%—

Frequently asked questions

Is Command R+ better than GPT-4?

Command R+ is the stronger model overall, scoring 32.4 to 29.1 on the Noometry Index.

Which is cheaper, Command R+ or GPT-4?

Command R+ is cheaper. It lists at $2.50 per million input tokens and $10 per million output tokens; GPT-4 lists at $30 and $60.

Is Command R+ or GPT-4 better for coding?

GPT-4 scores higher on coding benchmarks: 31.6 versus 29.1 in the Noometry coding category.

Which has the bigger context window?

Command R+ does, with 128K tokens against 8K.

How many benchmarks do Command R+ and GPT-4 share?

24 benchmarks have published results for both models. Command R+ has 34 scored results on Noometry and GPT-4 has 38.

Related comparisons

Go deeper