Noometry

Model comparison

Command R+ vs Gemma 2 27B

Command R+ is the stronger model overall, scoring 32.4 to 29.4 on the Noometry Index. Gemma 2 27B costs 6.7× less per token, which makes it the better buy when Command R+'s lead doesn't matter for your workload.

Last verified . 30 shared benchmarks.

Command R+ Cohere

32.4

Rank #257 Confirmed

Gemma 2 27B Google

29.4

Rank #312 Confirmed

Summary

  • They share 30 benchmarks with published results for both. Command R+ scores higher in 2 categories and Gemma 2 27B in 6 categories; 4 gaps are clear of the uncertainty.
  • The widest gap is in math, where Command R+ leads 28.9 to 10.7.
  • The biggest single-benchmark swing is LiveBench Coding: 19.1% for Command R+ and 36% for Gemma 2 27B.
  • Gemma 2 27B is cheaper at $0.65 / $0.65 per million input/output tokens, against $2.50 / $10 for Command R+.
  • Command R+ accepts more context: 128K tokens versus 8K.

Side by side

Command R+Gemma 2 27B
ProviderCohereGoogle
Noometry Index32.429.4
Released2024-08-302024-06-24
WeightsOpenOpen
Context window128K8K
Max output4K2K
Input $ / M tokens$2.50$0.65
Output $ / M tokens$10$0.65
Results tracked3434

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Gemma 2 27B leads

Command R+: 29.1 (#309), Gemma 2 27B: 34.1 (#246)

BenchmarkCommand R+Gemma 2 27B
BigCodeBench Instruct33.8%42.8%
LiveBench Coding19.1%36%
LMArena Coding11871211
BigCodeBench Complete41.9%52.5%
HumanEval+56.7%—
MBPP+63.5%—

Reasoning Gemma 2 27B leads

Command R+: 9.2 (#344), Gemma 2 27B: 15.3 (#315)

BenchmarkCommand R+Gemma 2 27B
LiveBench Reasoning24.8%28.1%
LMArena Hard Prompts11861198
DTBench54.9%48%
LiveBench Data Analysis38.1%47.9%
LMCA5%7.1%
Epoch Capabilities Index119.34122.08
LiveBench31.8%38.2%
SimpleBench17.4%—

Math Command R+ leads

Command R+: 28.9 (#242), Gemma 2 27B: 10.7 (#311)

BenchmarkCommand R+Gemma 2 27B
LiveBench Math21.3%26.5%
LMArena Math11881212
OTIS Mock AIME 2024-2025—1.4%
MATH Level 5—27.9%

Knowledge Command R+ leads

Command R+: 36.4 (#169), Gemma 2 27B: 19.0 (#280)

BenchmarkCommand R+Gemma 2 27B
LMArena Expert11741172
MMLU69.4%75.7%
GPQA Diamond—36.5%
Confabulations—27.1%
Vectara Hallucination Rate6.9%—

Multilingual Too close to call

Command R+: 38.6 (#227), Gemma 2 27B: 38.6 (#226)

BenchmarkCommand R+Gemma 2 27B
LMArena Non-English12161217
LMArena Chinese12261221
LMArena French12091247
LMArena German12161209
LMArena Japanese11661175
LMArena Korean11381174
LMArena Russian12271234
LMArena Spanish11891228

Instruction Following Too close to call

Command R+: 60.0 (#254), Gemma 2 27B: 60.5 (#249)

BenchmarkCommand R+Gemma 2 27B
LiveBench Instruction Following57.6%58.1%
LMArena Instruction Following11971206

Long Context Too close to call

Command R+: 37.3 (#219), Gemma 2 27B: 37.3 (#218)

BenchmarkCommand R+Gemma 2 27B
LMArena Longer Query12301231

Writing & Preference Too close to call

Command R+: 43.5 (#228), Gemma 2 27B: 44.2 (#225)

BenchmarkCommand R+Gemma 2 27B
LMArena Text12291231
LMArena Creative Writing12351241
LMArena Multi-Turn12131224
LiveBench Language29.7%32.6%

Frequently asked questions

Is Command R+ better than Gemma 2 27B?

Command R+ is the stronger model overall, scoring 32.4 to 29.4 on the Noometry Index. Gemma 2 27B costs 6.7× less per token, which makes it the better buy when Command R+'s lead doesn't matter for your workload.

Which is cheaper, Command R+ or Gemma 2 27B?

Gemma 2 27B is cheaper. It lists at $0.65 per million input tokens and $0.65 per million output tokens; Command R+ lists at $2.50 and $10.

Is Command R+ or Gemma 2 27B better for coding?

Gemma 2 27B scores higher on coding benchmarks: 34.1 versus 29.1 in the Noometry coding category.

Which has the bigger context window?

Command R+ does, with 128K tokens against 8K.

How many benchmarks do Command R+ and Gemma 2 27B share?

30 benchmarks have published results for both models. Command R+ has 34 scored results on Noometry and Gemma 2 27B has 34.

Related comparisons

Go deeper