Attribution
Data sources
The Noometry Index evaluates every model on 12,766 benchmark results from the sources below, which also supply all prices. Each score on the site links to the source it came from.
Last verified
| Source | License | Attribution |
|---|---|---|
| models.dev | MIT | Model metadata and pricing from models.dev (MIT). |
| Epoch AI Benchmarking Hub | CC-BY 4.0 | Epoch AI, 'Capabilities & benchmarking'. Published online at epoch.ai. Retrieved from https://epoch.ai/benchmarks |
| OpenRouter | Public API | Reseller pricing and catalog data from the OpenRouter public API. |
| LMArena | CC-BY 4.0 | Human-preference ratings from the LMArena leaderboard dataset on Hugging Face. |
| Kagi LLM Benchmark | Public leaderboard | Accuracy and output speed from the Kagi LLM Benchmarking Project. |
| HELM Capabilities | Apache 2.0 | Stanford CRFM, Holistic Evaluation of Language Models (HELM) Capabilities leaderboard. |
| Berkeley Function Calling Leaderboard | Public leaderboard | Tool-calling accuracy from the Berkeley Function Calling Leaderboard (BFCL), UC Berkeley Gorilla project. |
| Lech Mazur benchmarks | Public leaderboard | NYT Connections, Confabulations and Thematic Generalization leaderboards by Lech Mazur. |
| EvalPlus | Public leaderboard | HumanEval+ and MBPP+ pass@1 results from the EvalPlus leaderboard. |
| MathArena | Public leaderboard | Competition math accuracy from MathArena, ETH Zurich SRI Lab. |
| EQ-Bench | Public leaderboard | Elo ratings from EQ-Bench 4 and EQ-Bench Creative Writing v3 by Sam Paech. |
| BigCodeBench | Public leaderboard | Pass@1 results from the BigCodeBench leaderboard (BigCode project). |
| SWE-bench | Public leaderboard | Bash-only results from the official SWE-bench leaderboards (Princeton and Stanford). |
| τ²-bench | Public leaderboard | Tool-agent-user results from the τ²-bench leaderboard by Sierra Research. |
| Vectara Hallucination Leaderboard | Apache 2.0 | Summarization hallucination rates from Vectara's Hallucination Leaderboard (HHEM). |
| Model cards and launch posts | Quoted for reference | Self-reported results are stored with the source URL and the exact sentence they came from, and labelled "self-reported" wherever they appear. |
Corrections
If a number on Noometry is wrong, email contact@noometry.com with the page and the correct source. We usually fix sourcing errors within around a week and note material corrections on the editorial policy page.