Noometry

Knowledge benchmark

Vectara Hallucination Rate leaderboard

As of October 2026, GPT-5.4 nano has the lowest Vectara Hallucination Rate on Noometry at 3.1%, out of 96 models with results.

Last verified

About Vectara Hallucination Rate

How often a model adds claims that are not in the source document when asked to summarize it, judged by Vectara's HHEM model. Lower is better.

Category
Knowledge
Introduced
2023
Format
Grounded summary
Unit
Percent, lower is better
Official site
github.com

Top 15 models

  1. GPT-5.4 nano 3.1%
  2. Gemini 2.5 Flash-Lite 3.3%
  3. Phi-4 3.7%
  4. Llama-3.3-70B-Instruct 4.1%
  5. Gemma 3 12B 4.4%
  6. Mistral Large 4.5%
  7. Qwen3 8B 4.8%
  8. Amazon Nova Pro 5.1%
  9. Mistral Small 5.1%
  10. Nova 2 Lite 5.1%
  11. Gemma 4 26B A4B IT 5.2%
  12. Granite 4.0 H Small 5.2%
  13. DeepSeek-V3.2-Exp 5.3%
  14. Qwen3 14B 5.4%
  15. Amazon Nova Micro 5.5%

Sponsored placements are available on pages like this one. Advertise on Noometry

All results

#ModelProviderScoreSettingSourceDate
1GPT-5.4 nano OpenAI3.1%Vectara Hallucination Leaderboard
2Gemini 2.5 Flash-Lite Google3.3%Vectara Hallucination Leaderboard
3Phi-4 Microsoft3.7%Vectara Hallucination Leaderboard
4Llama-3.3-70B-Instruct Meta4.1%Vectara Hallucination Leaderboard
5Gemma 3 12B Google4.4%Vectara Hallucination Leaderboard
6Mistral Large Mistral AI4.5%Vectara Hallucination Leaderboard
7Qwen3 8B Alibaba (Qwen)4.8%Vectara Hallucination Leaderboard
8Amazon Nova Pro Amazon5.1%Vectara Hallucination Leaderboard
9Mistral Small Mistral AI5.1%Vectara Hallucination Leaderboard
10Nova 2 Lite Amazon5.1%Vectara Hallucination Leaderboard
11Gemma 4 26B A4B IT Google5.2%Vectara Hallucination Leaderboard
12Granite 4.0 H Small IBM5.2%Vectara Hallucination Leaderboard
13DeepSeek-V3.2-Exp DeepSeek5.3%Vectara Hallucination Leaderboard
14Qwen3 14B Alibaba (Qwen)5.4%Vectara Hallucination Leaderboard
15Amazon Nova Micro Amazon5.5%Vectara Hallucination Leaderboard
16DeepSeek-V3.1 DeepSeek5.5%Vectara Hallucination Leaderboard
17GPT-5.4 mini OpenAI5.5%Vectara Hallucination Leaderboard
18GPT-4.1 OpenAI5.6%Vectara Hallucination Leaderboard
19Qwen3-4B Alibaba (Qwen)5.7%Vectara Hallucination Leaderboard
20Grok 3 xAI5.8%Vectara Hallucination Leaderboard
21Qwen3 32B Alibaba (Qwen)5.9%Vectara Hallucination Leaderboard
22Amazon Nova Lite Amazon6.1%Vectara Hallucination Leaderboard
23DeepSeek-V3 DeepSeek6.1%Vectara Hallucination Leaderboard
24Gemma 3 4B Google6.4%Vectara Hallucination Leaderboard
25GPT-6 Sol OpenAI6.5%Vectara Hallucination Leaderboard
26Command R+ Cohere6.9%Vectara Hallucination Leaderboard
27Trinity Large Thinking Arcee AI6.9%Vectara Hallucination Leaderboard
28Gemini 2.5 Pro Google7%Vectara Hallucination Leaderboard
29GPT-5.4 OpenAI7%Vectara Hallucination Leaderboard
30Ministral 3B Mistral AI7.3%Vectara Hallucination Leaderboard
31Gemma 3 27B Google7.4%Vectara Hallucination Leaderboard
32Gemma 4 31B IT Google7.4%Vectara Hallucination Leaderboard
33Ministral 8B Mistral AI7.4%Vectara Hallucination Leaderboard
34Llama 4 Scout Meta7.7%Vectara Hallucination Leaderboard
35Gemini 2.5 Flash Google7.8%Vectara Hallucination Leaderboard
36Gemini 3.1 Flash Lite Google8.2%Vectara Hallucination Leaderboard
37Llama 4 Maverick Meta8.2%Vectara Hallucination Leaderboard
38GPT-5.4 Pro OpenAI8.3%Vectara Hallucination Leaderboard
39GPT-5.2 OpenAI8.4%Vectara Hallucination Leaderboard
40DeepSeek V4 Pro DeepSeek8.6%Vectara Hallucination Leaderboard
41GPT-6 Astra OpenAI8.7%Vectara Hallucination Leaderboard
42MiniMax-M2.5 MiniMax9.1%Vectara Hallucination Leaderboard
43Command A Cohere9.3%Vectara Hallucination Leaderboard
44GLM-4.5-Air Z.ai (Zhipu)9.3%Vectara Hallucination Leaderboard
45GLM-4.7-Flash Z.ai (Zhipu)9.3%Vectara Hallucination Leaderboard
46GPT-5.5 OpenAI9.3%Vectara Hallucination Leaderboard
47Qwen3 235B-A22B Alibaba (Qwen)9.3%Vectara Hallucination Leaderboard
48Qwen3-Next 80B-A3B Instruct Alibaba (Qwen)9.3%Vectara Hallucination Leaderboard
49C4ai Aya Expanse 8b Cohere9.5%Vectara Hallucination Leaderboard
50GLM-4.6 Z.ai (Zhipu)9.5%Vectara Hallucination Leaderboard
51GPT-4o OpenAI9.6%Vectara Hallucination Leaderboard
52Nemotron 3 Nano 30B A3B NVIDIA9.6%Vectara Hallucination Leaderboard
53Jamba Large AI21 Labs9.7%Vectara Hallucination Leaderboard
54Claude Haiku 4.5 Anthropic9.8%Vectara Hallucination Leaderboard
55GLM-5 Z.ai (Zhipu)10.1%Vectara Hallucination Leaderboard
56Claude Sonnet 4 Anthropic10.3%Vectara Hallucination Leaderboard
57Gemini 3.1 Pro Preview Google10.4%Vectara Hallucination Leaderboard
58GPT-5 Nano OpenAI10.5%Vectara Hallucination Leaderboard
59Qwen3.5 35B-A3B Alibaba (Qwen)10.5%Vectara Hallucination Leaderboard
60Qwen3.5-Flash Alibaba (Qwen)10.5%Vectara Hallucination Leaderboard
61Claude Sonnet 4.6 Anthropic10.6%Vectara Hallucination Leaderboard
62Qwen3.5 Plus Alibaba (Qwen)10.7%Vectara Hallucination Leaderboard
63Kimi K2.6 Moonshot AI10.8%Vectara Hallucination Leaderboard
64C4ai Aya Expanse 32b Cohere10.9%Vectara Hallucination Leaderboard
65Claude Opus 4.5 Anthropic10.9%Vectara Hallucination Leaderboard
66GPT-5.1 OpenAI10.9%Vectara Hallucination Leaderboard
67Qwen3.5 122B-A10B Alibaba (Qwen)11.2%Vectara Hallucination Leaderboard
68DeepSeek-R1 DeepSeek11.3%Vectara Hallucination Leaderboard
69GLM-4.7 Z.ai (Zhipu)11.7%Vectara Hallucination Leaderboard
70Claude Opus 4.1 Anthropic11.8%Vectara Hallucination Leaderboard
71MiniMax-M2.1 MiniMax11.8%Vectara Hallucination Leaderboard
72Claude Opus 4 Anthropic12%Vectara Hallucination Leaderboard
73Claude Opus 4.7 Anthropic12%Vectara Hallucination Leaderboard
74Claude Sonnet 4.5 Anthropic12%Vectara Hallucination Leaderboard
75Qwen3.5 27B Alibaba (Qwen)12.1%Vectara Hallucination Leaderboard
76Claude Opus 4.6 Anthropic12.2%Vectara Hallucination Leaderboard
77Mercury 2 Inception12.3%Vectara Hallucination Leaderboard
78GPT-5.6 Sol OpenAI12.4%Vectara Hallucination Leaderboard
79GPT-5 Mini OpenAI12.9%Vectara Hallucination Leaderboard
80MiniMax-M2.7 MiniMax12.9%Vectara Hallucination Leaderboard
81Gemini 3 Flash Preview Google13.5%Vectara Hallucination Leaderboard
82Gemini 3 Pro Google13.6%Vectara Hallucination Leaderboard
83gpt-oss-120b OpenAI14.2%Vectara Hallucination Leaderboard
84Kimi K2.5 Moonshot AI14.2%Vectara Hallucination Leaderboard
85Mistral Large 3 Mistral AI14.5%Vectara Hallucination Leaderboard
86GPT-5 OpenAI14.7%Vectara Hallucination Leaderboard
87Grok 4.1 Fast xAI17.8%Vectara Hallucination Leaderboard
88Kimi K2 (Jul 2025) Moonshot AI17.9%Vectara Hallucination Leaderboard
89o4-mini OpenAI18.6%Vectara Hallucination Leaderboard
90o4-mini OpenAI18.6%Vectara Hallucination Leaderboard
91Ministral 3 14B 2512 Mistral AI19.4%Vectara Hallucination Leaderboard
92Grok 4 Fast xAI19.7%Vectara Hallucination Leaderboard
93Ministral 3 8B 2512 (batch) Mistral AI21.7%Vectara Hallucination Leaderboard
94Mistral Medium Mistral AI22.7%Vectara Hallucination Leaderboard
95o3-pro OpenAI23.3%Vectara Hallucination Leaderboard
96Phi-4 Mini Microsoft23.5%Vectara Hallucination Leaderboard

Compare the leaders

Other knowledge benchmarks

Frequently asked questions

What does Vectara Hallucination Rate measure?

How often a model adds claims that are not in the source document when asked to summarize it, judged by Vectara's HHEM model. Lower is better.

Which model has the highest Vectara Hallucination Rate score?

As of October 2026, GPT-5.4 nano has the lowest Vectara Hallucination Rate on Noometry at 3.1%, out of 96 models with results.

What is the best open-weight model on Vectara Hallucination Rate?

Phi-4 has the lowest Vectara Hallucination Rate result among open-weight models at 3.7%, ranking 3 of 96 overall.