Noometry

Model comparison

Llama 3.1-70B vs Phi 3 Mini 128k Instruct

Llama 3.1-70B and Phi 3 Mini 128k Instruct score almost the same on the Noometry Index (29.6 vs 29.7), so choose on price, context window or the category you care about most.

Last verified . 19 shared benchmarks.

Llama 3.1-70B Meta

29.6

Rank #308 Confirmed

Phi 3 Mini 128k Instruct Microsoft

29.7

Rank #305 Confirmed

Summary

  • They share 19 benchmarks with published results for both. Llama 3.1-70B scores higher in 6 categories and Phi 3 Mini 128k Instruct in 2 categories; 8 gaps are clear of the uncertainty.
  • The widest gap is in math, where Phi 3 Mini 128k Instruct leads 31.6 to 13.5.
  • The biggest single-benchmark swing is BigCodeBench Instruct: 46.1% for Llama 3.1-70B and 29.6% for Phi 3 Mini 128k Instruct.

Side by side

Llama 3.1-70BPhi 3 Mini 128k Instruct
ProviderMetaMicrosoft
Noometry Index29.629.7
Released2024-07-232024-04-23
WeightsOpenOpen
Context window128K—
Max output4K—
Input $ / M tokens$0.40—
Output $ / M tokens$0.40—
Results tracked3519

Sponsored placements are available on pages like this one. Advertise on Noometry

Category by category

Coding Llama 3.1-70B leads

Llama 3.1-70B: 30.3 (#296), Phi 3 Mini 128k Instruct: 28.8 (#312)

BenchmarkLlama 3.1-70BPhi 3 Mini 128k Instruct
BigCodeBench Instruct46.1%29.6%
LMArena Coding12601039
BigCodeBench Complete54.8%40.6%
WeirdML9%—

Agentic & Tool Use Not comparable

Llama 3.1-70B: 25.1 (#112), Phi 3 Mini 128k Instruct: —

BenchmarkLlama 3.1-70BPhi 3 Mini 128k Instruct
TheAgentCompany6.9%—
BALROG27.9%—

Reasoning Llama 3.1-70B leads

Llama 3.1-70B: 21.6 (#220), Phi 3 Mini 128k Instruct: 19.6 (#256)

BenchmarkLlama 3.1-70BPhi 3 Mini 128k Instruct
LMArena Hard Prompts12411028
DTBench60%—
LMCA14.8%—
Epoch Capabilities Index125.92—

Math Phi 3 Mini 128k Instruct leads

Llama 3.1-70B: 13.5 (#304), Phi 3 Mini 128k Instruct: 31.6 (#222)

BenchmarkLlama 3.1-70BPhi 3 Mini 128k Instruct
LMArena Math12521089
OTIS Mock AIME 2024-20253.6%—
Omni-MATH21%—
MATH Level 536.7%—

Knowledge Phi 3 Mini 128k Instruct leads

Llama 3.1-70B: 24.2 (#269), Phi 3 Mini 128k Instruct: 26.8 (#254)

BenchmarkLlama 3.1-70BPhi 3 Mini 128k Instruct
LMArena Expert1209984
GPQA Diamond44.2%—
MMLU-Pro65.3%—
GPQA (HELM)42.6%—
MMLU80.1%—

Multilingual Llama 3.1-70B leads

Llama 3.1-70B: 38.8 (#225), Phi 3 Mini 128k Instruct: 25.2 (#285)

BenchmarkLlama 3.1-70BPhi 3 Mini 128k Instruct
LMArena Non-English12191000
LMArena Chinese12151016
LMArena French12611039
LMArena German12221006
LMArena Japanese1132899
LMArena Korean1140856
LMArena Russian12341004
LMArena Spanish12531059

Instruction Following Llama 3.1-70B leads

Llama 3.1-70B: 65.3 (#223), Phi 3 Mini 128k Instruct: 51.8 (#294)

BenchmarkLlama 3.1-70BPhi 3 Mini 128k Instruct
LMArena Instruction Following12311023
IFEval82.1%—

Long Context Llama 3.1-70B leads

Llama 3.1-70B: 37.6 (#214), Phi 3 Mini 128k Instruct: 30.4 (#289)

BenchmarkLlama 3.1-70BPhi 3 Mini 128k Instruct
LMArena Longer Query1241996

Writing & Preference Llama 3.1-70B leads

Llama 3.1-70B: 35.4 (#267), Phi 3 Mini 128k Instruct: 27.1 (#301)

BenchmarkLlama 3.1-70BPhi 3 Mini 128k Instruct
LMArena Text12611050
LMArena Creative Writing12321024
LMArena Multi-Turn1256989
EQ-Bench Creative Writing784—
WildBench75.8%—

Frequently asked questions

Is Llama 3.1-70B better than Phi 3 Mini 128k Instruct?

Llama 3.1-70B and Phi 3 Mini 128k Instruct score almost the same on the Noometry Index (29.6 vs 29.7), so choose on price, context window or the category you care about most.

Is Llama 3.1-70B or Phi 3 Mini 128k Instruct better for coding?

Llama 3.1-70B scores higher on coding benchmarks: 30.3 versus 28.8 in the Noometry coding category.

How many benchmarks do Llama 3.1-70B and Phi 3 Mini 128k Instruct share?

19 benchmarks have published results for both models. Llama 3.1-70B has 35 scored results on Noometry and Phi 3 Mini 128k Instruct has 19.

Related comparisons

Go deeper