Knowledge benchmark

# Confabulations leaderboard

> Confabulations results for 51 AI models, led by GPT-5 at 10.3%. What the benchmark measures, who runs it, and a source for every score.
- Canonical page: https://noometry.com/benchmarks/confabulations
- Last updated: 2026-10-10
- Title: Confabulations Leaderboard (October 2026): Scores by Model

As of October 2026, GPT-5 has the lowest Confabulations on Noometry at 10.3%, out of 51 models with results.

Last verified October 10, 2026

## About Confabulations

Questions about provided documents where the answer is deliberately missing. The score averages the rate of made-up answers and the rate of refusing questions that do have answers. Lower is better.

- **Category:** [Knowledge](https://noometry.com/best/knowledge)
- **Introduced:** 2024
- **Format:** Grounded questions
- **Unit:** Percent, lower is better
- **Official site:** [github.com](https://github.com/lechmazur/confabulations)

## Top 15 models

Top models on Confabulations

1.  GPT-5 10.3%
2.  Gemini 2.5 Pro 10.6%
3.  Grok-3 mini 10.8%
4.  GLM-4.5 11.3%
5.  o1 11.7%
6.  Qwen3-30B-A3B 12.3%
7.  Grok 4 12.4%
8.  Gemini 2.0 Flash (Feb 2025) 12.4%
9.  DeepSeek-R1 12.7%
10.  Claude Sonnet 4 13.2%
11.  GPT-5 Mini 13.3%
12.  Gemini 1.5 Pro (May 2024) 13.5%
13.  GPT-4.5 13.6%
14.  Grok 3 14.2%
15.  o3-pro 14.2%
16.  9101112131415

Sponsored placements are available on pages like this one. [Advertise on Noometry](https://noometry.com/advertise)

## All results

Confabulations results by model
| # | Model | Provider | Score | Setting | Source | Date |
| --- | --- | --- | --- | --- | --- | --- |
| 1 | [GPT-5](https://noometry.com/models/gpt-5) | [OpenAI](https://noometry.com/providers/openai) | 10.3% | medium reasoning | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 2 | [Gemini 2.5 Pro](https://noometry.com/models/gemini-2-5-pro) |  [![](/logos/google.svg) Google](https://noometry.com/providers/google) | 10.6% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 3 | [Grok-3 mini](https://noometry.com/models/grok-3-mini) | [xAI](https://noometry.com/providers/xai) | 10.8% | high | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 4 | [GLM-4.5](https://noometry.com/models/glm-4-5) | [Z.ai (Zhipu)](https://noometry.com/providers/zai) | 11.3% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 5 | [o1](https://noometry.com/models/o1) | [OpenAI](https://noometry.com/providers/openai) | 11.7% | medium reasoning | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 6 | [Qwen3-30B-A3B](https://noometry.com/models/qwen3-30b-a3b) |  [![](/logos/alibaba.svg) Alibaba (Qwen)](https://noometry.com/providers/alibaba) | 12.3% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 7 | [Grok 4](https://noometry.com/models/grok-4) | [xAI](https://noometry.com/providers/xai) | 12.4% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 8 | [Gemini 2.0 Flash (Feb 2025)](https://noometry.com/models/gemini-2-0-flash) |  [![](/logos/google.svg) Google](https://noometry.com/providers/google) | 12.4% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 9 | [DeepSeek-R1](https://noometry.com/models/deepseek-r1) |  [![](/logos/deepseek.svg) DeepSeek](https://noometry.com/providers/deepseek) | 12.7% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 10 | [Claude Sonnet 4](https://noometry.com/models/claude-sonnet-4) | [Anthropic](https://noometry.com/providers/anthropic) | 13.2% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 11 | [GPT-5 Mini](https://noometry.com/models/gpt-5-mini) | [OpenAI](https://noometry.com/providers/openai) | 13.3% | medium reasoning | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 12 | [Gemini 1.5 Pro (May 2024)](https://noometry.com/models/gemini-1-5-pro) |  [![](/logos/google.svg) Google](https://noometry.com/providers/google) | 13.5% | sept | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 13 | [GPT-4.5](https://noometry.com/models/gpt-4-5) | [OpenAI](https://noometry.com/providers/openai) | 13.6% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 14 | [Grok 3](https://noometry.com/models/grok-3) | [xAI](https://noometry.com/providers/xai) | 14.2% | no reasoning | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 15 | [o3-pro](https://noometry.com/models/o3-pro) | [OpenAI](https://noometry.com/providers/openai) | 14.2% | medium reasoning | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 16 | [o3](https://noometry.com/models/o3) | [OpenAI](https://noometry.com/providers/openai) | 14.4% | high reasoning | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 17 | [Claude 3.7 Sonnet](https://noometry.com/models/claude-3-7-sonnet) | [Anthropic](https://noometry.com/providers/anthropic) | 14.7% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 18 | [GPT-4o](https://noometry.com/models/gpt-4o) | [OpenAI](https://noometry.com/providers/openai) | 15.3% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 19 | [QwQ-32B](https://noometry.com/models/qwq-32b) |  [![](/logos/alibaba.svg) Alibaba (Qwen)](https://noometry.com/providers/alibaba) | 15.6% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 20 | [Qwen3 235B-A22B](https://noometry.com/models/qwen3-235b-a22b) |  [![](/logos/alibaba.svg) Alibaba (Qwen)](https://noometry.com/providers/alibaba) | 15.6% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 21 | [gpt-oss-120b](https://noometry.com/models/gpt-oss-120b) | [OpenAI](https://noometry.com/providers/openai) | 15.7% | medium reasoning | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 22 | [o4-mini](https://noometry.com/models/o4-mini) | [OpenAI](https://noometry.com/providers/openai) | 15.8% | high reasoning | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 23 | [Claude Opus 4](https://noometry.com/models/claude-opus-4) | [Anthropic](https://noometry.com/providers/anthropic) | 15.9% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 24 | [Chatgpt 4o Latest 20250326](https://noometry.com/models/chatgpt-4o) | [OpenAI](https://noometry.com/providers/openai) | 16.6% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 25 | [Gemini 2.5 Flash](https://noometry.com/models/gemini-2-5-flash) |  [![](/logos/google.svg) Google](https://noometry.com/providers/google) | 16.8% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 26 | [Claude Opus 4.1](https://noometry.com/models/claude-opus-4-1) | [Anthropic](https://noometry.com/providers/anthropic) | 17.1% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 27 | [Llama 3.1-405B](https://noometry.com/models/llama-3-1-405b) |  [![](/logos/meta.svg) Meta](https://noometry.com/providers/meta) | 17.6% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 28 | [o3-mini](https://noometry.com/models/o3-mini) | [OpenAI](https://noometry.com/providers/openai) | 17.9% | medium reasoning | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 29 | [Gemini 2.0 Pro](https://noometry.com/models/gemini-2-0-pro) |  [![](/logos/google.svg) Google](https://noometry.com/providers/google) | 18.4% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 30 | [o1-mini](https://noometry.com/models/o1-mini) | [OpenAI](https://noometry.com/providers/openai) | 18.6% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 31 | [Qwen2.5 72B Instruct](https://noometry.com/models/qwen2-5-72b-instruct) |  [![](/logos/alibaba.svg) Alibaba (Qwen)](https://noometry.com/providers/alibaba) | 19.1% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 32 | [Claude 3.5 Sonnet](https://noometry.com/models/claude-3-5-sonnet) | [Anthropic](https://noometry.com/providers/anthropic) | 19.9% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 33 | [Grok-2 (Dec 2024)](https://noometry.com/models/grok-2) | [xAI](https://noometry.com/providers/xai) | 20.1% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 34 | [Kimi K2 (Jul 2025)](https://noometry.com/models/kimi-k2) | [Moonshot AI](https://noometry.com/providers/moonshot) | 20.4% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 35 | [Mistral Large](https://noometry.com/models/mistral-large) |  [![](/logos/mistral.svg) Mistral AI](https://noometry.com/providers/mistral) | 21.4% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 36 | [Qwen2.5-Max](https://noometry.com/models/qwen2-5-max) |  [![](/logos/alibaba.svg) Alibaba (Qwen)](https://noometry.com/providers/alibaba) | 21.8% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 37 | [Mistral Medium 3](https://noometry.com/models/mistral-medium-3) |  [![](/logos/mistral.svg) Mistral AI](https://noometry.com/providers/mistral) | 21.9% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 38 | [Llama 4 Maverick](https://noometry.com/models/llama-4-maverick) |  [![](/logos/meta.svg) Meta](https://noometry.com/providers/meta) | 22.6% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 39 | [Claude 3 Opus](https://noometry.com/models/claude-3-opus) | [Anthropic](https://noometry.com/providers/anthropic) | 22.7% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 40 | [Llama-3.3-70B-Instruct](https://noometry.com/models/llama-3-3-70b-instruct) |  [![](/logos/meta.svg) Meta](https://noometry.com/providers/meta) | 22.8% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 41 | [MiniMax-01](https://noometry.com/models/minimax-01) |  [![](/logos/minimax.svg) MiniMax](https://noometry.com/providers/minimax) | 23.9% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 42 | [Mistral Small 3](https://noometry.com/models/mistral-small-3) |  [![](/logos/mistral.svg) Mistral AI](https://noometry.com/providers/mistral) | 25.2% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 43 | [DeepSeek-V3](https://noometry.com/models/deepseek-v3) |  [![](/logos/deepseek.svg) DeepSeek](https://noometry.com/providers/deepseek) | 26.1% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 44 | [Gemma 2 27B](https://noometry.com/models/gemma-2-27b) |  [![](/logos/google.svg) Google](https://noometry.com/providers/google) | 27.1% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 45 | [GPT-4 Turbo](https://noometry.com/models/gpt-4-turbo) | [OpenAI](https://noometry.com/providers/openai) | 28.4% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 46 | [Phi-4](https://noometry.com/models/phi-4) |  [![](/logos/microsoft.svg) Microsoft](https://noometry.com/providers/microsoft) | 29.4% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 47 | [Amazon Nova Pro](https://noometry.com/models/amazon-nova-pro) | [Amazon](https://noometry.com/providers/amazon) | 30.1% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 48 | [Claude 3 Haiku](https://noometry.com/models/claude-3-haiku) | [Anthropic](https://noometry.com/providers/anthropic) | 34.2% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 49 | [Claude 3.5 Haiku](https://noometry.com/models/claude-3-5-haiku) | [Anthropic](https://noometry.com/providers/anthropic) | 36.7% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 50 | [GPT-4o mini](https://noometry.com/models/gpt-4o-mini) | [OpenAI](https://noometry.com/providers/openai) | 37.2% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |
| 51 | [Gemma 3 27B](https://noometry.com/models/gemma-3-27b) |  [![](/logos/google.svg) Google](https://noometry.com/providers/google) | 40.3% |  | [Lech Mazur benchmarks](https://github.com/lechmazur/confabulations) |  |

## Compare the leaders

-   [GPT-5 vs Gemini 2.5 Pro](https://noometry.com/compare/gemini-2-5-pro-vs-gpt-5)
-   [GPT-5 vs Grok-3 mini](https://noometry.com/compare/gpt-5-vs-grok-3-mini)
-   [GPT-5 vs GLM-4.5](https://noometry.com/compare/glm-4-5-vs-gpt-5)
-   [GPT-5 vs o1](https://noometry.com/compare/gpt-5-vs-o1)
-   [Gemini 2.5 Pro vs Grok-3 mini](https://noometry.com/compare/gemini-2-5-pro-vs-grok-3-mini)
-   [Gemini 2.5 Pro vs GLM-4.5](https://noometry.com/compare/gemini-2-5-pro-vs-glm-4-5)

## Other knowledge benchmarks

-   [GPQA Diamond](https://noometry.com/benchmarks/gpqa-diamond)
-   [Humanity's Last Exam](https://noometry.com/benchmarks/hle)
-   [SimpleQA Verified](https://noometry.com/benchmarks/simpleqa-verified)
-   [MMLU-Pro](https://noometry.com/benchmarks/mmlu-pro)
-   [Vectara Hallucination Rate](https://noometry.com/benchmarks/vectara-hallucination)
-   [LMArena Expert](https://noometry.com/benchmarks/arena-expert)
-   [GPQA (HELM)](https://noometry.com/benchmarks/helm-gpqa)
-   [ARC (AI2) Challenge](https://noometry.com/benchmarks/arc-challenge) (reference)
-   [BoolQ](https://noometry.com/benchmarks/boolq) (reference)
-   [MMLU](https://noometry.com/benchmarks/mmlu) (reference)
-   [OpenBookQA](https://noometry.com/benchmarks/openbookqa) (reference)
-   [TriviaQA](https://noometry.com/benchmarks/triviaqa) (reference)

## Frequently asked questions

### What does Confabulations measure?

Questions about provided documents where the answer is deliberately missing. The score averages the rate of made-up answers and the rate of refusing questions that do have answers. Lower is better.

### Which model has the highest Confabulations score?

As of October 2026, GPT-5 has the lowest Confabulations on Noometry at 10.3%, out of 51 models with results.

### What is the best open-weight model on Confabulations?

GLM-4.5 has the lowest Confabulations result among open-weight models at 11.3%, ranking 4 of 51 overall.

### Cite this page

Noometry. (2026). Confabulations leaderboard. Retrieved October 10, 2026, from https://noometry.com/benchmarks/confabulations

Quote Noometry with a link back to this page. It is also available in [Markdown](https://noometry.com/md/benchmarks/confabulations.md).