Noometry

Model comparison

Command R+ vs Grok-2 (Dec 2024)

Grok-2 (Dec 2024) is the stronger model overall, scoring 33.4 to 32.0 on the Noometry Index.

Last verified . 27 shared benchmarks.

Command R+ Cohere

32.0

Rank #251 Confirmed

Grok-2 (Dec 2024) xAI

33.4

Rank #233 Confirmed

Summary

  • They share 27 benchmarks with published results for both. Command R+ scores higher in 2 categories and Grok-2 (Dec 2024) in 6 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in reasoning, where Grok-2 (Dec 2024) leads 16.4 to 8.9.
  • The biggest single-benchmark swing is LiveBench Math: 21.3% for Command R+ and 54.9% for Grok-2 (Dec 2024).
  • Command R+ has downloadable open weights; the other is API-only.

Side by side

Command R+Grok-2 (Dec 2024)
ProviderCoherexAI
Noometry Index32.033.4
Released2024-08-302024-08-13
WeightsOpenProprietary
Context window128K—
Max output4K—
Input $ / M tokens$2.50—
Output $ / M tokens$10—
Results tracked3434

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Grok-2 (Dec 2024) leads

Command R+: 28.6 (#304), Grok-2 (Dec 2024): 32.9 (#254)

BenchmarkCommand R+Grok-2 (Dec 2024)
LiveBench Coding19.1%46.4%
LMArena Coding11871288
WeirdML—22.2%
BigCodeBench Instruct33.8%—
BigCodeBench Complete41.9%—
HumanEval+56.7%—
MBPP+63.5%—

Reasoning Grok-2 (Dec 2024) leads

Command R+: 8.9 (#341), Grok-2 (Dec 2024): 16.4 (#294)

BenchmarkCommand R+Grok-2 (Dec 2024)
SimpleBench17.4%22.7%
LiveBench Reasoning24.8%54.8%
LMArena Hard Prompts11861272
DTBench54.9%65.2%
LiveBench Data Analysis38.1%54.5%
Epoch Capabilities Index119.28130.48
LiveBench31.8%54.3%
LMCA5%—

Math Command R+ leads

Command R+: 26.6 (#239), Grok-2 (Dec 2024): 19.6 (#278)

BenchmarkCommand R+Grok-2 (Dec 2024)
LiveBench Math21.3%54.9%
LMArena Math11881283
OTIS Mock AIME 2024-2025—11.5%
MATH Level 5—63.5%
FrontierMath (Feb 2025 set)—0.7%

Knowledge Command R+ leads

Command R+: 35.9 (#165), Grok-2 (Dec 2024): 29.8 (#225)

BenchmarkCommand R+Grok-2 (Dec 2024)
LMArena Expert11741254
GPQA Diamond—53.8%
Confabulations—20.1%
Vectara Hallucination Rate6.9%—
MMLU69.4%—

Multilingual Grok-2 (Dec 2024) leads

Command R+: 38.8 (#222), Grok-2 (Dec 2024): 43.3 (#183)

BenchmarkCommand R+Grok-2 (Dec 2024)
LMArena Non-English12161282
LMArena Chinese12261289
LMArena French12081317
LMArena German12161287
LMArena Japanese11661244
LMArena Korean11381237
LMArena Russian12271286
LMArena Spanish11891281

Instruction Following Grok-2 (Dec 2024) leads

Command R+: 60.2 (#249), Grok-2 (Dec 2024): 67.1 (#197)

BenchmarkCommand R+Grok-2 (Dec 2024)
LiveBench Instruction Following57.6%69.6%
LMArena Instruction Following11971270

Long Context Grok-2 (Dec 2024) leads

Command R+: 37.3 (#214), Grok-2 (Dec 2024): 38.7 (#186)

BenchmarkCommand R+Grok-2 (Dec 2024)
LMArena Longer Query12311276

Writing & Preference Grok-2 (Dec 2024) leads

Command R+: 43.7 (#223), Grok-2 (Dec 2024): 48.8 (#192)

BenchmarkCommand R+Grok-2 (Dec 2024)
LMArena Text12291305
LMArena Creative Writing12351284
LMArena Multi-Turn12131290
LiveBench Language29.7%45.6%
Short-Story Creative Writing—63.6%

Frequently asked questions

Is Command R+ better than Grok-2 (Dec 2024)?

Grok-2 (Dec 2024) is the stronger model overall, scoring 33.4 to 32.0 on the Noometry Index.

Is Command R+ or Grok-2 (Dec 2024) better for coding?

Grok-2 (Dec 2024) scores higher on coding benchmarks: 32.9 versus 28.6 in the Noometry coding category.

How many benchmarks do Command R+ and Grok-2 (Dec 2024) share?

27 benchmarks have published results for both models. Command R+ has 34 scored results on Noometry and Grok-2 (Dec 2024) has 34.

Related comparisons

Go deeper