Noometry

Model comparison

Codellama 34b Instruct vs Llama 3.1-70B

Codellama 34b Instruct is the stronger model overall, scoring 30.8 to 29.6 on the Noometry Index.

Last verified . 12 shared benchmarks.

Codellama 34b Instruct Meta

30.8

Rank #287 Confirmed

Llama 3.1-70B Meta

29.6

Rank #308 Confirmed

Summary

  • They share 12 benchmarks with published results for both. Codellama 34b Instruct scores higher in 1 category and Llama 3.1-70B in 6 categories; 7 gaps are clear of the uncertainty.
  • The widest gap is in math, where Codellama 34b Instruct leads 31.0 to 13.5.
  • The biggest single-benchmark swing is BigCodeBench Complete: 37.1% for Codellama 34b Instruct and 54.8% for Llama 3.1-70B.

Side by side

Codellama 34b InstructLlama 3.1-70B
ProviderMetaMeta
Noometry Index30.829.6
Released—2024-07-23
WeightsOpenOpen
Context window—128K
Max output—4K
Input $ / M tokens—$0.40
Output $ / M tokens—$0.40
Results tracked1435

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 3.1-70B leads

Codellama 34b Instruct: 28.5 (#314), Llama 3.1-70B: 30.3 (#296)

BenchmarkCodellama 34b InstructLlama 3.1-70B
BigCodeBench Instruct29%46.1%
LMArena Coding10461260
BigCodeBench Complete37.1%54.8%
WeirdML—9%
HumanEval+43.9%—
MBPP+56.3%—

Agentic & Tool Use Not comparable

Codellama 34b Instruct: —, Llama 3.1-70B: 25.1 (#112)

BenchmarkCodellama 34b InstructLlama 3.1-70B
TheAgentCompany—6.9%
BALROG—27.9%

Reasoning Llama 3.1-70B leads

Codellama 34b Instruct: 19.6 (#255), Llama 3.1-70B: 21.6 (#220)

BenchmarkCodellama 34b InstructLlama 3.1-70B
LMArena Hard Prompts10321241
DTBench—60%
LMCA—14.8%
Epoch Capabilities Index—125.92

Math Codellama 34b Instruct leads

Codellama 34b Instruct: 31.0 (#230), Llama 3.1-70B: 13.5 (#304)

BenchmarkCodellama 34b InstructLlama 3.1-70B
LMArena Math10561252
OTIS Mock AIME 2024-2025—3.6%
Omni-MATH—21%
MATH Level 5—36.7%

Knowledge Not comparable

Codellama 34b Instruct: —, Llama 3.1-70B: 24.2 (#269)

BenchmarkCodellama 34b InstructLlama 3.1-70B
GPQA Diamond—44.2%
MMLU-Pro—65.3%
GPQA (HELM)—42.6%
LMArena Expert—1209
MMLU—80.1%

Multilingual Llama 3.1-70B leads

Codellama 34b Instruct: 25.8 (#284), Llama 3.1-70B: 38.8 (#225)

BenchmarkCodellama 34b InstructLlama 3.1-70B
LMArena Non-English10111219
LMArena Chinese9761215
LMArena French—1261
LMArena German—1222
LMArena Japanese—1132
LMArena Korean—1140
LMArena Russian—1234
LMArena Spanish—1253

Instruction Following Llama 3.1-70B leads

Codellama 34b Instruct: 52.2 (#291), Llama 3.1-70B: 65.3 (#223)

BenchmarkCodellama 34b InstructLlama 3.1-70B
LMArena Instruction Following10281231
IFEval—82.1%

Long Context Llama 3.1-70B leads

Codellama 34b Instruct: 30.9 (#284), Llama 3.1-70B: 37.6 (#214)

BenchmarkCodellama 34b InstructLlama 3.1-70B
LMArena Longer Query10131241

Writing & Preference Llama 3.1-70B leads

Codellama 34b Instruct: 28.2 (#297), Llama 3.1-70B: 35.4 (#267)

BenchmarkCodellama 34b InstructLlama 3.1-70B
LMArena Text10661261
LMArena Creative Writing10321232
LMArena Multi-Turn10151256
EQ-Bench Creative Writing—784
WildBench—75.8%

Frequently asked questions

Is Codellama 34b Instruct better than Llama 3.1-70B?

Codellama 34b Instruct is the stronger model overall, scoring 30.8 to 29.6 on the Noometry Index.

Is Codellama 34b Instruct or Llama 3.1-70B better for coding?

Llama 3.1-70B scores higher on coding benchmarks: 30.3 versus 28.5 in the Noometry coding category.

How many benchmarks do Codellama 34b Instruct and Llama 3.1-70B share?

12 benchmarks have published results for both models. Codellama 34b Instruct has 14 scored results on Noometry and Llama 3.1-70B has 35.

Related comparisons

Go deeper