CalibratedAgentsLLM hallucination detection (2026): 13 techniques, benchmarks and best practices
Every technique, where each one fails, and what to use.
By Calibrated Agents Research Team · 3 Nov 2026 · Updated 3 Nov 2026 · 28 min read
13 techniques · three signals
13ways to catch a model making things up, each blind in a predictable place
- 02–06Belief: reads the modelMisses errors it is sure about
- 01 · 13Opinion: a second model or a personBiased, or slow and costly
- 07–12Evidence: checks the sourceCan say what is wrong
Thirteen families of technique try to catch a model making things up, from LLM-as-a-judge and SelfCheckGPT to semantic entropy, NLI models and span detectors. Each is blind in a predictable place. This guide maps them, the benchmark evidence on where they break, and which to use for RAG, agents and review work.
What is LLM hallucination detection?
LLM hallucination detection flags statements a model produced that are false or not supported by its sources. Confidence-based methods read what the model believes; evidence-based methods check each claim against documents. For grounded work such as RAG, support and policy answers, the evidence check is the one that holds up, and it should name the exact part that is wrong.
FIGURE 1
Flagging an answer is easy. Finding the wrong part is not.
F1 on RAGTruth: does this answer contain an error, and which words are wrong?
What is an LLM hallucination?
A hallucination is a fluent statement that is false or not supported by the sources the model was given.
OpenAI's working definition is "plausible but false statements generated by language models" (OpenAI, Sep 2025). The research literature splits the problem along two axes, and the split decides which detector can help.
The first axis is what the output is checked against. A factuality hallucination conflicts with the world. A faithfulness hallucination diverges from the instruction, the supplied context, or the answer's own earlier statements (Huang et al., ACM TOIS). The second axis is the kind of failure. An intrinsic error contradicts the source. An extrinsic error adds something the source cannot verify (Ji et al.).
The labels are not settled. Vectara's FaithBench adds a benign label for an added fact that readers find acceptable (FaithBench). Meta's HalluLens redefines extrinsic as inconsistent with the model's training data (HalluLens, ACL 2025). Stanford's legal study adds misgrounded: a correct statement that cites a source which does not support it (Magesh et al., 2024).
The practical point: unsupported and wrong are different properties. A RAG grounding check asks whether the context supports a claim, not whether the claim is true. Any detector, and any vendor, should say which one it measures.
Why do LLMs hallucinate?
Models hallucinate because training and evaluation reward guessing, because no model can learn every fact, and because retrieval narrows the problem without closing it.
Hallucination is a rate to manage, not a bug to fix once.
OpenAI's Kalai, Nachum, Vempala and Zhang argue that accuracy-only evaluation works like a multiple-choice exam with no penalty for wrong answers. Guessing scores sometimes; saying "I don't know" never does (arXiv 2509.04664). Their SimpleQA comparison shows the cost.
FIGURE 2
A model that never abstains buys 2 points of accuracy with 49 points of errors.
Share of SimpleQA questions, by outcome
Theory points the same way. Xu, Jain and Kankanhalli show that a computable model cannot learn every computable function, so a general-purpose model must hallucinate on some inputs (Xu et al., 2024). Huang et al. catalogue the mechanisms by stage: long-tail and outdated data, fine-tuning on answers beyond the model's knowledge, RLHF sycophancy, and decoding that drifts away from the context (Huang et al.).
Retrieval helps but does not cure it. Stanford found that Lexis+ AI, Westlaw AI-Assisted Research and Ask Practical Law AI each hallucinated 17–33% of the time, despite "hallucination-free" marketing (Magesh et al., May 2024). Reasoning models are not a reliable fix either: OpenAI's system card reports a PersonQA hallucination rate of 33% for o3 against 16% for o1 (o3 and o4-mini system card, Apr 2025).
How often do LLMs hallucinate?
Between about 2% and 80% of the time, depending on the task. On Vectara's grounded-summarisation leaderboard (as of its May 2026 update, scored by Vectara's own HHEM-2.3), the best model sits at 1.8%, GPT-5 High at 15.1% and o3-pro at 23.3% (Vectara leaderboard). On closed-book SimpleQA, o3 hallucinated on 51% and o4-mini on 79% (system card). Google DeepMind's FACTS suite found no model above 70% overall in January 2026 (InfoQ, Jan 2026).
The spread matters for detection. The same detector shows very different precision on a workload with a 3% error rate than on one with 45%, so a published accuracy number tells you little until you know the base rate behind it.
Why does hallucination detection matter now?
Because courts and regulators already hold the deployer responsible for what its model says.
Every incident below involved a specific claim that could have been checked against a document before it reached someone.
- Moffatt v. Air Canada (14 Feb 2024). The chatbot invented a post-travel bereavement refund. The tribunal rejected the argument that the bot was a separate legal entity and held the airline liable for negligent misrepresentation (ABA Business Law Today).
- Mata v. Avianca (2023). Lawyers who filed ChatGPT-invented cases were fined $5,000 (Bloomberg Law). Damien Charlotin's tracker counts 11 such US cases in 2023, 37 in 2024, 524 in 2025 and 668 in 2026 by late July (MSBA, 27 Jul 2026).
- Deloitte Australia (Oct 2025) agreed to repay part of an A$440,000 government contract after its report carried a fabricated court quote and references to works that do not exist (AP).
- Cursor (Apr 2025). An AI support agent told users about a login policy that did not exist, and subscribers cancelled in public (Fortune).
Frameworks point at verification. NIST AI 600-1 names "confabulation" as a generative-AI risk and suggests verifying sources and citations before deployment and monitoring after it (actions MS-2.5-003 and MG-4.1-002) (NIST). India's RBI FREE-AI framework for the financial sector (13 Aug 2025) sets out 7 Sutras and 26 recommendations, including governance, audit and incident reporting (RBI). Under a provisional EU AI Act Omnibus deal, high-risk accuracy obligations move to 2 December 2027 for Annex III systems (Gibson Dunn, May 2026).
What types of hallucination detection are there?
There are three types, by the signal they read. Belief methods read the model's own confidence or consistency. Opinion methods ask another model or a person. Evidence methods check the output against source documents.
They also differ in how finely they point and in the access they need.
| Question | Options | Why it matters |
|---|---|---|
| What signal? | belief the model's confidence or consistency · evidence a check against sources · opinion a second model or a person | Belief signals miss errors the model is sure about. Evidence signals miss errors when the source itself is wrong. |
| How fine? | Response · sentence · claim · span | A response score says something is wrong. A span says what to fix. |
| What access? | Black-box text · token probabilities · model weights | Closed APIs restrict probabilities and never expose weights. |
FIGURE 3
Where each family of detectors reads its signal
Numbers refer to the 13 techniques below
Belief · reads the model
04Token probabilities, stated confidence
06Probes on internal states
02 05Resampling: SelfCheckGPT, semantic entropy
03Self-verification
Opinion · reads the output
01LLM-as-a-judge
13Human review
Evidence · compares output with source
07 08 09NLI, grounding detectors, span detectors
10 11 12Claim decomposition, rules, cloud APIs
Evidence methods tie each finding to a passage in the source. Only belief and opinion methods work when there is no source to check against.
13 methods for detecting LLM hallucinations
There are 13 commonly used approaches to LLM hallucination detection, in three groups. Belief-based methods (02–06) estimate whether the model is uncertain or inconsistent. Evidence-based methods (07–12) compare claims with source documents. Opinion-based methods (01, 13) ask another model or a person.
For source-grounded work such as RAG, evidence-based verification is usually the most relevant, because it tests whether each claim is supported.
The 13 techniques at a glance
| Technique | What it measures | Main limitation | Best application |
|---|---|---|---|
| LLM-as-a-judge | Another model's judgement | Bias; near chance on hard errors | Rubric grading, offline comparison |
| Sampling consistency | Agreement across resampled answers | Misses consistent errors | Closed-book generation, offline |
| Self-verification | The model re-checking its claims | Can lower accuracy without outside feedback | Closed-book lists, with another check |
| Token probabilities | Model confidence | Confidence is not correctness | Cheap triage feature |
| Semantic entropy | Spread of meanings across samples | Misses confident errors; 5–10× cost | Short closed-book QA |
| Hidden-state probes | Signals inside the model | Does not transfer; needs weights | Own open-weight model, one task |
| NLI / entailment | Whether the source entails a claim | Copy-heavy errors pass | Short-claim screening |
| Small grounding detectors | Support for a sentence or response | Domain shift, tables | Low-latency RAG guarding |
| Span detectors | Which words are unsupported | Localising is hard | Highlighting RAG answers |
| Claim decomposition | Each atomic claim vs knowledge or web | Decomposition noise, cost | Offline long-form evaluation |
| Rule-based checks | Deterministic constraints | Only what the rules encode | Numbers, dates, IDs, citations |
| Cloud grounding APIs | Support for a response or claim | Thresholds left to you | Managed per-turn guard |
| Human review | Expert judgement | Cost, drift, scale | High-stakes cases, calibration labels |
Each entry below gives the mechanism, the measured failure, and the situation it fits. Numbers a vendor measured on its own product or benchmark are marked vendor-reported.
01LLM-as-a-judge (Ragas, DeepEval, G-Eval)
- opinion
- response or claim
- black-box
- How it works
A strong LLM grades an output against a rubric or a context. RAG libraries wrap judges in claim pipelines: Ragas scores the share of claims the context supports, while DeepEval counts a claim as faithful unless the context contradicts it. Two tools both called "faithfulness" therefore measure different things.
- Where it fails
Hard, natural errors. GPT-4o scored 86.5% on the partly synthetic HaluBench (Lynx paper) but 56.3% balanced accuracy on FaithBench, where chance is 50% (FaithBench). Judges show position bias (GPT-4 kept its verdict on swapped order only 65% of the time), verbosity bias, and preference for their own text (Zheng et al.; Panickssery et al.). Only 3 of 13 judges aligned closely with humans (Thakur et al.). GPT-4o's calibration error on JudgeBench ran 39–47 points (Tian et al.).
- Use it when
You grade tone, helpfulness or style with no ground truth, or compare models on averages. A cheap lexical check ranked nine models almost as well as GPT-4 Turbo, so averages forgive judge error; blocking a single answer does not. Our post on why LLM-as-a-judge fails in production works through the base-rate problem behind this.
FIGURE 4
On hard, natural errors, prompted judges land near a coin flip.
Balanced accuracy on FaithBench (660 summaries from 10 modern LLMs, chosen where detectors disagreed)
02Sampling consistency (SelfCheckGPT)
- belief
- sentence
- black-box, resampling
- How it works
Draw several extra answers and score each sentence of the original by how well the samples agree with it. Facts the model knows recur; inventions drift. With 20 samples on GPT-3 biographies, SelfCheckGPT reached 93.4 sentence-level AUC-PR (Manakul et al., 2023).
- Where it fails
Consistent errors pass, because agreement measures stability, not truth. On RAG data it scored 58.8 response-level F1 on RAGTruth (LettuceDetect paper), and the ChainPoll ensemble scored 46.7, below a single prompt (vendor-reported, Luna paper). It needs N extra generations from the same model, which a cached or third-party answer cannot give.
- Use it when
There are no reference documents, the generator can be re-queried, and the check can run offline.
03Self-verification (Chain-of-Verification, self-correction)
- belief
- claim
- black-box, multi-call
- How it works
The model drafts an answer, writes verification questions, answers them independently and revises. CoVe reduced hallucinations on list questions and biographies (Dhuliawala et al., 2024).
- Where it fails
Without outside feedback, self-correction often makes answers worse: GPT-3.5's CommonSenseQA accuracy fell from 75.8 to 38.1 after one round (Huang et al., ICLR 2024). A TACL survey finds it works only with reliable external feedback or large-scale fine-tuning (Kamoi et al., 2024).
- Use it when
You generate closed-book lists or biographies and can afford several calls, paired with an external check. Never as the only check.
04Token probabilities (logprobs) and stated confidence
- belief
- token to answer
- logprobs or black-box
- How it works
Read the model's token log-probabilities, perplexity or entropy, or ask it how sure it is. Nearly free at generation time.
- Where it fails
Confidence is not correctness: OpenAI's cookbook shows a model about 99% confident in a wrong answer (OpenAI Cookbook). On RAG faithfulness, perplexity scored an AUC of 0.509 (ReDeEP). Stated confidence tends to be overconfident (Xiong et al., ICLR 2024), though RLHF models can state it better than their token probabilities show (Tian et al., 2023). Logprob access on newer reasoning models is restricted.
- Use it when
Outputs are short labels, or as one cheap feature among others for triage.
05Semantic entropy
- belief
- answer
- black-box, resampling
- How it works
Sample several answers, group them by meaning, and measure entropy over the groups. High entropy means the model is guessing (Farquhar et al., Nature 2024).
- Where it fails
Its authors scope it to confabulations, the arbitrary errors. The CHOKE study shows models giving wrong answers with high certainty after small prompt changes, an error class that uncertainty cannot see (Simhi et al., 2025). It costs about 5–10 times one generation (Kossen et al.), and AUROC on long-form biographies is only 0.64–0.72 (Kang et al., 2025).
- Use it when
You answer short, closed-book questions and want to catch the model guessing.
06Hidden-state and attention probes (SAPLMA, Lookback Lens, ReDeEP)
- belief
- token to answer
- model weights
- How it works
Train a small classifier on the model's internal activations (SAPLMA), or measure how much attention goes back to the context (Lookback Lens) or how much parametric knowledge overrides it (ReDeEP). One forward pass, near-zero cost.
- Where it fails
Transfer. Lookback Lens scores 85–87 AUROC in-domain but about 66 on a new task. SAPLMA reaches 0.704 AUC on RAGTruth and 0.537 on Dolly. Error detectors "fail to generalize across datasets" (Orgad et al., ICLR 2025), and once a spurious correlation in RAGTruth is removed, methods approach random out of distribution (Dubanowska et al., 2025). Closed APIs rule them out.
- Use it when
You serve your own open-weight model on one stable task and can label 1–2k in-domain examples.
FIGURE 5
Confidence signals sit near chance on RAG faithfulness.
AUC for detecting unfaithful answers on RAGTruth, LLaMA2-7B
07NLI and entailment models (AlignScore, HHEM)
- evidence
- pair
- black-box
- How it works
A trained classifier scores whether a premise (the source) entails a hypothesis (the claim). AlignScore (355M) matched GPT-4-based metrics on 22 datasets. Vectara's HHEM-2.1-Open runs in under 600 MB of RAM and takes about 1.5 s for 2k tokens on a CPU.
- Where it fails
Lexical shortcuts. Evaluators miss unsupported claims that copy heavily from the source and penalise supported claims that paraphrase it; detectors with similar accuracy agreed on fewer than half their flags on 5 of 14 datasets (Godbole & Jia, 2025). Compound claims that are half supported get mislabelled. Most open models are English-only.
- Use it when
You need cheap, high-volume English screening of short claims against one passage.
08Small grounding detectors (MiniCheck, Granite Guardian, Lynx, Luna)
- evidence
- sentence or response
- black-box
- How it works
Compact models trained for groundedness. MiniCheck (770M) checks each sentence and reaches GPT-4-level accuracy on LLM-AggreFact at about 400 times lower cost. Granite Guardian and Lynx return a response-level verdict, Lynx with a written rationale. Luna (440M) handles 16k tokens in under a second on an L4 (vendor-reported).
- Where it fails
Domain shift and numbers. Luna scored 0.64 AUROC on financial reasoning over tables and kept about 68% of its performance past 16k tokens. FinanceBench was the weakest HaluBench subset for most systems. MiniCheck has no separate contradiction label and supports one document at a time. On LLM-AggreFact, 0.4–0.8B models sit only 4–5 points below 27B leaders around 79.5, so the field is compressed.
- Use it when
You guard a latency-sensitive RAG product close to the detector's training domain.
09Span-level hallucination detectors (LettuceDetect)
- evidence
- span
- black-box
- How it works
A token classifier marks which words of the answer the context does not support. LettuceDetect (ModernBERT) processes 30–60 examples a second on one GPU, with a 4,096-token cap.
- Where it fails
Localisation is hard (Figure 1): 79.2 F1 at example level becomes 58.9 at span level. Top RAGTruth systems are trained on RAGTruth's own training split and tested on the same distribution. On relabelled data, a strong system still reaches only 47.5% span-level F1 (RT4CHART, 2026). It marks where, not why.
- Use it when
You want to highlight unsupported text in RAG answers within the detector's training domain. For why that level of detail matters, see why answer-level hallucination detection isn't enough.
10Claim decomposition and fact-checking (FActScore, SAFE, VeriScore)
- evidence
- atomic claim
- black-box, multi-call
- How it works
Split long text into atomic claims and verify each against a knowledge source or web search. FActScore estimates within 2% of human scores on biographies; Google DeepMind's SAFE agrees with crowd raters on about 72% of facts at more than 20 times lower cost.
- Where it fails
The decomposition adds its own errors. On the same biographies, FActScore moved from 33.0% to 46.0% depending only on how claims were given context (DnDScore). VeriScore found search support for only about 55% of claims and cost about $1,038 per 400 generations with GPT-4o (Song et al., 2024), so a low score can mean weak retrieval, not false claims.
- Use it when
You evaluate long-form, open-domain text offline, with no fixed source document.
11Rule-based checks
- evidence
- field
- any
- How it works
Deterministic code matches numbers, dates, entities and citations against the source, or validates a schema. It says exactly which rule broke.
- Where it fails
It catches only what the rules encode, and no study we found measures it. The indirect case for it is strong: tables, finance and arithmetic are where learned detectors are weakest, and users found code-based checks easier to trust than LLM judges (Shankar et al., UIST 2024).
- Use it when
Outputs carry numbers, amounts, dates, IDs or citations. Run it next to a learned check, not instead of one.
12Cloud grounding APIs (Bedrock, Azure, Google) and guardrails
- evidence
- response to claim
- black-box
- How it works
Amazon Bedrock returns a grounding and a relevance score per response. Azure AI Content Safety offers a fast yes/no mode and a slower mode that explains ungrounded segments. Google's check-grounding returns per-claim citations with offsets and targets under 500 ms. NeMo Guardrails wraps self-checks, sampling, AlignScore and Lynx.
- Where it fails
Thresholds are left to you and move with the product tier: AWS says its two tiers score the same response differently. Bedrock does not say what is wrong. Azure is optimised for English. In NeMo, a verdict truncated at the token limit scores 0.0 and blocks a valid answer. A retrieval miss always reads as "ungrounded".
- Use it when
You already run inside that cloud and need a managed per-turn guard.
13Human review
- opinion
- any
- any
- How it works
People check outputs, usually on a sample. Hamel Husain recommends binary pass/fail verdicts with written critiques from one domain expert, then tuning automated checks until they agree with that expert (Husain).
- Where it fails
Humans drift too. Graders changed their own criteria as they saw more outputs (Shankar et al.), and FaithBench's expert annotators reached only Krippendorff's α 0.58 on hard three-way labels. It does not scale to full traffic.
- Use it when
Stakes are high, for the uncertain band an automated check routes to people, and to produce the labels that keep automated thresholds calibrated.
A newer category, decision-model APIs, returns a probability over a fixed set of answers instead of free text. Microsoft-Decision-1, OpenAI's Decisions API, TypeSafe AI's Jev and Cloudflare's Clef all work this way, and libraries such as DeepEval now offer faithfulness modes built on them. In our benchmark of the four as support checkers on 296 LLM-AggreFact answers, the first three tied on accuracy (AUROC 0.84–0.86) and differed mainly on latency and cost; Clef-flash trailed. We ran that test ourselves, from one location, so it is not independent, and we found no third-party evaluation of these models for hallucination detection.
Hallucination detection methods compared
No technique scores well on every column.
Pick the requirements you cannot give up to narrow the list. Evidence-based methods are the only ones that can say what is wrong; belief-based methods are the only ones that need no source.
| Technique | Signal | Granularity | Needs a source? | Closed API OK? | Says what is wrong? | Cost per check | Main failure |
|---|---|---|---|---|---|---|---|
| LLM-as-a-judge | Opinion | Response or claim | For grounding | Yes | Unvalidated rationale | Seconds, cents | ~56% on FaithBench; biased, poorly calibrated |
| Sampling consistency | Belief | Sentence | No | If resampled | No | N extra generations | Consistent errors pass |
| Self-verification | Belief | Claim | No | Yes | Partly | Several calls | Can lower accuracy |
| Token probabilities | Belief | Token to answer | No | Often restricted | No | ~Free | AUC 0.509 on RAG |
| Semantic entropy | Belief | Answer | No | Yes | No | 5–10× generation | Blind to confident errors |
| Hidden-state probes | Belief | Token to answer | No | No | No | ~Free after training | Does not transfer |
| NLI / entailment | Evidence | Pair | Yes | Yes | No | ~1.5 s CPU per 2k tokens | Copy-but-wrong passes |
| Small grounding detectors | Evidence | Sentence or response | Yes | Yes | Mostly no | Sub-second on GPU | Tables, domain shift |
| Span detectors | Evidence | Span | Yes | Yes | Where, not why | 30–60 per second per GPU | Span F1 far below answer F1 |
| Claim decomposition | Evidence | Claim | Knowledge or web | Yes | Yes | ~$2.60 per generation | Decomposition noise |
| Rule-based checks | Evidence | Field | Depends | Yes | Names the rule | Negligible | Only what rules encode |
| Cloud grounding APIs | Evidence | Response to claim | Yes | Yes | Varies by vendor | Google targets <500 ms | Threshold tuning left to you |
| Human review | Opinion | Any | Ideally | Yes | Yes | Highest | Drift, does not scale |
Claim decomposition cost derived from VeriScore's ~$1,038 per 400 generations with GPT-4o. Latency figures are the ones reported in each source cited above.
Which hallucination detection method should you use?
Start from what you have to check against and what you will do with the result.
If you hold the source, check against it. If you will act on a single answer, you need a calibrated threshold and the location of the error, not an average.
| Your situation | Use first | Add | Avoid relying on |
|---|---|---|---|
| RAG, support or policy answers, checked before they reach a customer | Evidence-based check at claim or span level, calibrated on your labels | Rule checks for numbers and dates; human review for the uncertain band | Logprobs, perplexity, a single judge verdict |
| QA or audit team re-checking work people or models already produced | Evidence check that ranks items by likely error and shows where | Expert labels to recalibrate as standards drift | Random sampling alone |
| Numbers, tables, contracts, statements, invoices | Rule checks plus evidence check | Per-domain calibration | Small detectors trained on prose |
| Open-domain long-form text with no source | Claim decomposition with search, offline | Sampling consistency | Any single-pass score |
| Short closed-book answers | Semantic entropy or sampling | Stated confidence as a feature | Self-correction without feedback |
| Your own open-weight model on one stable task | Hidden-state or attention probes | An evidence check for anything new | Probes on a new domain |
| Tone, helpfulness, brand voice | LLM judge with a binary rubric, tuned to one expert | Periodic human spot-checks | Judge scores on a 0–100 scale |
| Comparing models or prompts offline | LLM judge averages | A small labelled set | Single-example verdicts |
How to detect hallucinations in your LLM application, step by step
Decide what each output is checked against, label a few hundred real outputs, choose the technique for your situation, set the threshold at your real error rate, and send the uncertain band to people.
Each step below closes a failure described earlier.
- Name the reference. Source documents, a policy, a database, or nothing at all. This decides between evidence-based and belief-based methods.
- Label real outputs. Take a few hundred outputs from your own traffic, not a public benchmark, and mark the exact parts that are wrong. Keep errors at the rate they actually occur.
- Choose by situation. Use the table above, and add rule checks wherever numbers, dates or IDs appear.
- Set a threshold and measure it. Pick the operating point from your labels, report precision and recall at that point, and check calibration, not only AUROC.
- Route and relabel. Send the uncertain band to reviewers and feed their verdicts back as labels, because models, prompts and reviewers' own criteria drift.
How do you evaluate hallucination detection models?
Evaluate a detector on your own naturally generated, relabelled outputs, at your real error rate and a fixed threshold, and read public benchmarks only at the granularity you will run at.
Three problems make headline scores misleading.
Benchmarks measure different things
RAGTruth has about 18,000 natural RAG responses labelled at word level. HaluEval is mostly synthetic and response-level. LLM-AggreFact pools 11 human-labelled datasets at claim level with a live leaderboard. FaithBench keeps only cases where detectors disagreed, so it is hard by design. MTRAG tests multi-turn RAG, and PHANTOM tests long-context finance.
Labels are noisier than the leaderboards suggest
Two third-party relabellings show how much the original labels missed. RAGTruth++ re-annotated 408 samples, and the count of hallucinated spans rose from 86 to 865. RAGTruth-Enhance relabelled 2,675 samples and found 1.68 times more hallucinated cases. Both figures are as summarised in RT4CHART (2026). Detectors that catch real but unlabelled errors are penalised. A spurious correlation in RAGTruth explains much of the state of the art (Dubanowska et al.), and uncertainty methods drop by up to 45.9% when ROUGE-based labels are replaced with better ones (Janiak et al., 2025).
Error rate, threshold and calibration decide whether a number transfers
Accuracy and F1 depend on how common errors are. One preprint shows 91.6% accuracy alongside a 50% false-positive rate on a dataset where most examples are hallucinated (Sinha, 2025). AUROC ignores the base rate but says nothing about the threshold you will ship; the base-rate problem is the same one that sinks LLM judges in production. Calibration is the least reported property of all: in one field-extraction study, calibration error was 0.237 on a new domain and fell to 0.025 only after recalibrating on that domain's labels (arXiv 2606.24420, 2026).
The rule: test on your own naturally generated, relabelled data, at your real error rate, at a fixed threshold, and report calibration error next to AUROC. Treat benchmarks built by a detector's own developer with matching caution.
What do practitioners say about hallucination detection?
Engineers distrust raw scores and lean on citations, binary rubrics and people.
Forum discussion is more sceptical than vendor marketing.
- In a 227-point Hacker News thread, commenters advised splitting judgements into several binary or small-enum questions instead of a 0–100 score, and said calibration only means something against labelled data and can break when a vendor updates its model (HN, Oct 2026).
- On semantic entropy, commenters flagged the roughly tenfold cost and false alarms on questions with many valid answers; none reported running it in production (HN, Jun 2024).
- A sceptic noted that a fully general detector would be a truth oracle, and that internally consistent fabrications such as fake court cases defeat heuristics (HN, Jun 2025).
- One production team tried several eval platforms, then fell back on custom scripts and weekly spreadsheet reviews (HN, Jun 2025). A RAG builder said the hard part is diagnosing why an answer failed (HN, Jul 2025).
- After the Cursor incident, the most-upvoted lesson was to keep people in the loop for customer-facing answers, because an AI's confident tone makes its errors harder to spot (HN, Apr 2025).
Vendors converge on the same mitigations. Anthropic's guidance is to allow "I don't know", extract direct quotes first, and retract any claim without a supporting quote (Claude docs). Document-extraction company Extend replaced logprob confidence with a second-pass review agent (vendor post, Extend, Jan 2026). The tooling market is moving too: Galileo, Cleanlab and Arize have all been acquired.
What makes an effective LLM hallucination verifier?
The evidence above adds up to a specification.
No single source describes a system with all of these properties; each one closes a gap a technique above leaves open.
- Evidence first. Check each claim against the supplied sources, because confidence misses errors the model is sure about.
- The exact part that is wrong. Point to the span in the output and the passage in the source, so the fix takes seconds.
- Three outcomes, not two. Separate "contradicted by the source" from "not found in the source", so a retrieval gap does not read as a lie.
- Calibrated on your data. Probabilities that mean what they say for your domain, recalibrated from your own labels as models and standards drift.
- Numbers handled deterministically. Values, dates, entities and citations checked by rules next to learned scoring.
- Fast and black-box. One pass, no logprobs, no weights, cheap enough to run on every output.
- Safe failure. When the check cannot run, it says so; it never returns a silent pass or a silent block.
- Audit trail and routing. A record of every check, with the uncertain band sent to people.
Three limits apply even then. A verifier inherits the quality of its sources: if the source is wrong, a grounded wrong answer passes. The accuracy ceiling is set by human agreement, which is low on hard cases. And evidence-based checking outside English is barely measured anywhere.
Try it yourself: verify a RAG answer
The fastest way to judge any hallucination detector is to run one real answer and its source through it and see what comes back.
Here is the shape of a useful result.
FIGURE 6
One answer, its source, and what a useful result should identify.
SOURCE DOCUMENT
The refund request must be submitted within 30 days of the original transaction. Refunds are processed within 7 business days after approval.
GENERATED ANSWER
Customers can request a refund within 60 days within 30 days of the original transaction. Once approved, refunds are processed within 7 business days.
CONTRADICTED“within 60 days”: the source says 30 days. Fix: replace with “within 30 days”.
SUPPORTED“within 7 business days”: matches the source’s processing timeline.
Then do the same with an output from your own pipeline, no synthetic examples needed, and examine:
- Which parts are supported, and which contradict the source?
- Which parts the source simply does not cover?
- What evidence backs each finding, and could a reviewer act on it in seconds?
- Which outputs and parts does the verifier rank as most likely to be wrong?
How does Ground verify outputs against source documents?
Ground can be used for hallucination detection wherever there is source evidence to check against.
Ground, the first product from CalibratedAgents, is an independent verification layer built on one principle: the system producing an output shouldn't be its only judge.
Send Ground an output and the source it should have used. It returns the exact part that is wrong, why it is wrong, the evidence, and the fix. It ranks the parts of an output most likely to contain errors, so reviewers start where it matters. It learns your standard from about 200 of your own labelled examples, without fine-tuning a foundation model, and returns a check in under a second. It works on AI or human-written output alike.
From your side, the workflow has three steps:
- Send the output and its source, through the console or the API.
- Get back an overall risk score and, for each flagged part, what is wrong, why, where the evidence sits in the source, and what to do next.
- Label about 200 of your own examples so Ground learns your standard.
In our RAGTruth++ benchmark, Ground calibrated on 50 labelled answers led a tuned LLM judge given the same labels by 4.0 points of balanced accuracy at locating hallucinated text (77.6 vs 73.6); on whole-answer verdicts the two tied. We ran that benchmark ourselves, so treat it as a claim to test on your own data.
| Verifier property | How Ground handles it |
|---|---|
| Evidence first | Checks the output against the source you send, not against the model's confidence |
| The exact part that is wrong | Returns what is wrong, where, why, and what to do next, with supporting evidence |
| Calibrated on your data | Learns your standard from about 200 labelled examples |
| Fast and black-box | Under a second a check; needs only the output and its source |
| Safe failure | When a check cannot run, returns an explicit unavailable status instead of a silent pass |
| Routing to people | Ranks outputs and parts by likely error so QA, audit and review teams re-check the right ones first |
When Ground is not the right tool
- There is no source. Ground verifies against evidence. For open-domain facts with nothing to check against, use claim decomposition with search or sampling methods.
- The question is taste. Tone, helpfulness and brand voice have no evidence to check. A rubric judge tuned to an expert is the better fit.
- The source is wrong. Ground checks faithfulness to what you give it. Fix retrieval and source quality alongside verification.
QA, audit and review teams are trialling Ground to re-check the right conversations first, inside the review budget they already have.
Put your own verifier to the test
Take one answer from your RAG pipeline and the documents it was given. Run them through Ground and compare what it flags with what your current checks catch.
Run an output in the consoleRead the API docs
Want us to run it on a sample of your data? Write to [email protected].
How this guide was put together
- Sources reviewed in October and November 2026: peer-reviewed papers and preprints from 2022–2026, model cards, vendor documentation, leaderboards, Hacker News threads and news reporting on incidents. Every number links to its source.
- Numbers a vendor measured on its own product or benchmark are marked vendor-reported, and benchmarks we ran ourselves are labelled as ours. Leaderboard entries are self-submitted and read as ties within about a point; the LLM-AggreFact standings are as read on 3 Nov 2026.
- Disclosure: CalibratedAgents builds Ground, an evidence-based verifier, so we have a stake in the conclusions; we have kept every claim about other methods to what their sources measure. The field-extraction calibration study cited above (arXiv 2606.24420) is by a CalibratedAgents co-founder.
- This guide will be updated as new methods and benchmarks are published. The date at the top reflects the last update.
Key research papers on hallucination detection
These are the papers most of this guide rests on, each linked to its original publication.
The full source list, including documentation and incident reporting, is at the end.
| Paper | What it investigates | Why it matters |
|---|---|---|
| Why language models hallucinate OpenAI, 2025 | Training and evaluation incentives | Explains why models guess instead of abstaining |
| Hallucination is inevitable Xu et al., 2024 | Theoretical limits of LLMs | No model can remove hallucination entirely |
| RAGTruth Niu et al., ACL 2024 | Word-level labels on ~18,000 natural RAG answers | The standard span-level RAG benchmark; labels later found incomplete |
| HaluEval Li et al., EMNLP 2023 | A large, mostly synthetic benchmark | Early scale; easier negatives than natural errors |
| LLM-AggreFact Tang et al., 2024 | 11 human-labelled grounding datasets | Live leaderboard for claim-level checkers |
| FaithBench Bao et al., NAACL 2025 | Hard summaries where detectors disagree | Shows detectors near chance on natural errors |
| SelfCheckGPT Manakul et al., EMNLP 2023 | Consistency across sampled answers | Black-box detection without sources |
| Semantic entropy Farquhar et al., Nature 2024 | Uncertainty over meanings | Catches confabulations; misses confident errors |
| CHOKE Simhi et al., 2025 | Wrong answers given with high certainty | Why uncertainty signals miss a class of errors |
| ReDeEP Sun et al., ICLR 2025 | How models use retrieved context | Context-use signals beat plain uncertainty on RAG |
| MiniCheck Tang, Laban, Durrett, 2024 | Small sentence-level fact checkers | GPT-4-level grounding at about 400 times lower cost |
| LettuceDetect Kovács & Recski, 2025 | Token-level span detection | Measures the gap between flagging and locating |
| FActScore Min et al., EMNLP 2023 | Atomic-claim factuality | Defined claim decomposition for long-form text |
| LLMs cannot self-correct reasoning yet Huang et al., ICLR 2024 | Models correcting their own answers | Self-review without feedback can hurt |
Key terms in hallucination detection
- Hallucination
- A fluent statement from a model that is false or not supported by its sources.
- Confabulation
- A hallucination that is arbitrary: the model gives a different wrong answer each time it is asked. NIST uses the term for the whole risk.
- Faithfulness (groundedness)
- Whether an output is supported by the context the model was given, such as retrieved documents.
- Factuality
- Whether an output is true about the world, regardless of the context.
- Span-level detection
- Marking the exact words of an output that are unsupported, rather than scoring the whole response.
- Calibration
- How well a detector's stated probabilities match how often it is right. A calibrated 0.8 is right about 80% of the time.
- Balanced accuracy
- The average of the detection rate on hallucinated and on faithful outputs, so a lopsided dataset cannot inflate it. 50% is chance.
- AUROC
- How well a score ranks hallucinated outputs above faithful ones, across all thresholds. 0.5 is chance, 1.0 is perfect.
Frequently asked questions
What is hallucination detection?
Hallucination detection flags statements in a model's output that are false or not supported by the sources it was given. Methods differ in the signal they read: the model's own confidence, agreement between samples, a second model's judgement, or a direct check of each claim against source evidence.
How do you detect hallucinations in LLM outputs?
Start by deciding what each output should be checked against. With source documents, check each claim against them with an evidence-based verifier and calibrate it on your own labelled outputs. Without a source, use sampling consistency or claim decomposition with web search. Add rule checks for numbers and send uncertain cases to people.
How does hallucination detection work in RAG systems?
In RAG, the retrieved documents are the reference. A detector checks each claim in the answer against them and marks claims the documents contradict or do not support, and its threshold is calibrated on a few hundred of your own labelled answers. Confidence signals help little here: perplexity scored an AUC of 0.509 on RAGTruth in the ReDeEP study.
Can LLM confidence scores detect hallucinations?
Only partly. Token probabilities and stated confidence measure what the model believes, not whether its source supports the answer. OpenAI's own cookbook shows a model about 99% confident in a wrong answer, and on RAG faithfulness perplexity scored close to chance. Confidence works best as one cheap feature next to an evidence check.
Can an LLM-as-a-judge detect hallucinations reliably?
Only on easy cases. GPT-4o scored 86.5% on the partly synthetic HaluBench but 56.3% balanced accuracy on FaithBench, which uses hard, naturally occurring errors. Judges also show position, verbosity and self-preference bias and are poorly calibrated, so they suit rubric grading and offline comparison more than gating single answers.
Does semantic entropy detect all hallucinations?
No. Its authors scope it to confabulations, the arbitrary errors where a model answers differently each time. A model that repeats the same wrong fact in every sample looks certain, and studies show models can be wrong with high certainty. It also costs roughly 5 to 10 times the compute of one generation.
What is the difference between factuality and faithfulness?
Factuality asks whether a statement is true about the world. Faithfulness asks whether the context the model was given, such as retrieved documents or a policy, supports it. Most production checks measure faithfulness, so a true statement the source never mentions can still be flagged as unsupported.
What is the difference between hallucination detection and fact-checking?
Fact-checking verifies claims against the world, usually with search or a knowledge base. Hallucination detection is broader: in RAG and document work it usually checks faithfulness, meaning whether the sources the model was given support each claim. Claim-decomposition tools such as FActScore and SAFE sit where the two meet.
What is the difference between claim-level and span-level detection?
Claim-level detection splits an output into atomic claims and judges each one; span-level detection marks the exact words that are unsupported. Spans are harder: on RAGTruth, LettuceDetect scores 79.2 F1 at example level and 58.9 at span level, and GPT-4-turbo falls from 63.4 to 28.3. Claim splitting adds noise of its own.
What is the best open-source hallucination detection model?
There is no single best. On the LLM-AggreFact leaderboard, small open models such as MiniCheck-Flan-T5-L (0.8B, 75.0) and FactCG-DeBERTa-L (0.4B, 75.6) sit within about 5 points of the 27B leaders, and LettuceDetect marks unsupported spans in RAG answers. All degrade on domains unlike their training data, so test on your own outputs.
How many labelled examples do you need to calibrate a hallucination detector?
Usually a few hundred from your own traffic. Thresholds and probabilities shift between domains: in one extraction study, calibration error fell from 0.237 to 0.025 only after recalibrating on the new domain's labels. Ground learns a customer's standard from about 200 labelled examples, while trained probes such as Lookback Lens need around 1,000 to 2,000.
Can hallucination detection work on human-written outputs?
Evidence-based checks can, because they compare text with its source regardless of who wrote it. QA and audit teams can use the same check on reports, support replies and reviews written by people. Confidence and probe methods cannot, because they need access to the model that produced the text.
Can hallucination detection run in real time?
Some methods can. Small grounding detectors and NLI models run in roughly a second or less, Google targets under 500 ms for its grounding check, and Ground returns a check in under a second. Sampling, semantic entropy, claim decomposition and human review add extra model calls or people, so they suit offline or asynchronous checks.
How do you measure precision and recall for a hallucination detector?
Fix a threshold, run the detector on labelled outputs at your real error rate, and count. Precision is the share of flags that are real errors; recall is the share of real errors that get flagged. Report balanced accuracy or AUROC and calibration error alongside, because plain accuracy shifts with how common errors are.
Can CalibratedAgents Ground be used for hallucination detection?
Yes, where there is source evidence to check against. Ground verifies an output against its sources, returns the exact part that is wrong with the reason and the evidence, ranks where errors are most likely, learns your standard from about 200 labelled examples and returns a check in under a second. It works on AI or human-written output.
Sources
Definitions, causes and prevalence
- Huang et al., A Survey on Hallucination in Large Language Models, ACM TOIS (v2 Nov 2024)
- Ji et al., Survey of Hallucination in Natural Language Generation (2022, v7 Jul 2024)
- Zhang et al., Siren's Song in the AI Ocean (Sep 2023)
- Bang et al., HalluLens, ACL 2025
- Kalai, Nachum, Vempala, Zhang, Why language models hallucinate, OpenAI (5 Sep 2025); arXiv 2509.04664
- Xu, Jain, Kankanhalli, Hallucination is Inevitable (Jan 2024)
- Magesh et al., Hallucination-Free? Assessing the Reliability of Leading AI Legal Research Tools, Stanford (May 2024)
- OpenAI, o3 and o4-mini system card (16 Apr 2025)
- Vectara, Hallucination Leaderboard (updated 11 May 2026)
- InfoQ, FACTS Benchmark Suite (12 Jan 2026)
Judges, sampling and self-verification
- Zheng et al., Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena (v4 Dec 2023)
- Thakur et al., Judging the Judges (v5 Jan 2025)
- Panickssery, Bowman, Feng, LLM Evaluators Recognize and Favor Their Own Generations (Apr 2024)
- Tian et al., judge calibration on JudgeBench (Aug 2025)
- Ragas, Faithfulness metric; DeepEval, Faithfulness metric
- Manakul, Liusie, Gales, SelfCheckGPT (Mar 2023)
- Dhuliawala et al., Chain-of-Verification, Findings of ACL 2024
- Huang et al., Large Language Models Cannot Self-Correct Reasoning Yet, ICLR 2024
- Kamoi et al., When Can LLMs Actually Correct Their Own Mistakes?, TACL 2024
Uncertainty and internal states
- OpenAI Cookbook, Using logprobs (Dec 2023)
- Xiong et al., Can LLMs Express Their Uncertainty?, ICLR 2024; Tian et al., Just Ask for Calibration, EMNLP 2023
- Farquhar et al., Detecting hallucinations using semantic entropy, Nature (Jun 2024); Kossen et al., Semantic Entropy Probes (Jun 2024)
- Simhi et al., Trust Me, I'm Wrong (CHOKE) (2025)
- Kang et al., uncertainty quantification survey and benchmark (Oct 2025)
- Azaria & Mitchell, The Internal State of an LLM Knows When It's Lying, Findings of EMNLP 2023
- Orgad et al., LLMs Know More Than They Show, ICLR 2025
- Chuang et al., Lookback Lens, EMNLP 2024; Sun et al., ReDeEP, ICLR 2025
- Multi-signal confidence for LLM document field extraction, arXiv 2606.24420 (Jun 2026)
Evidence-based detectors
- Zha et al., AlignScore, ACL 2023; Vectara, HHEM model card
- Tang, Laban, Durrett, MiniCheck (v2 Oct 2024); LLM-AggreFact leaderboard
- Padhi et al., Granite Guardian (Dec 2024); Ravi et al., Lynx (Jul 2024); Belyi et al., Luna (Jun 2024)
- Kovács & Recski, LettuceDetect (Feb 2025)
- Godbole & Jia, benchmarking attribution evaluators (Jan 2025)
- Min et al., FActScore, EMNLP 2023; Wei et al., Long-form factuality (SAFE), NeurIPS 2024; Song, Kim, Iyyer, VeriScore (Jun 2024); Wanner et al., DnDScore (Dec 2024)
- Shankar et al., Who Validates the Validators?, UIST 2024
- AWS, Bedrock contextual grounding check; Microsoft, Groundedness detection; Google Cloud, Check grounding; NVIDIA, NeMo Guardrails fact-checking
- Eden AI, Jev: a new kind of AI model (updated 22 Oct 2026)
Benchmarks and evaluation
- Niu et al., RAGTruth, ACL 2024; Li et al., HaluEval, EMNLP 2023; Bao et al., FaithBench, NAACL 2025
- Katsis et al., MTRAG (Jan 2025); PHANTOM, NeurIPS 2025
- RT4CHART and RAGTruth relabelling (v2 Jul 2026)
- Dubanowska et al., spurious correlations in hallucination detection, EMNLP 2025 Findings; Janiak et al., The Illusion of Progress (Aug 2025); Sinha, conformal hallucination detection (Dec 2025)
Our related posts
- Hallucination detection in 2026: why answer-level detection isn't enough
- Hallucination detection benchmark 2026 (RAGTruth++)
- Decision models benchmarked (2026)
- Why LLM-as-a-judge fails in production: the base rate problem
- Why outputs need independent verification
Practice, incidents and regulation
- Hacker News threads: LLM confidence scoring (Oct 2026), semantic entropy (Jun 2024), Show HN detector (Jun 2025), eval tooling (Jun 2025), RAG in production (Jul 2025), Cursor support bot (Apr 2025)
- Hamel Husain, Creating an LLM-as-a-judge that drives business results; Anthropic, Reduce hallucinations; Extend, Review Agent (Jan 2026)
- ABA Business Law Today, Moffatt v. Air Canada (Feb 2024); Bloomberg Law, Mata v. Avianca fine; MSBA, hard numbers on AI phantom precedents (27 Jul 2026); AP, Deloitte partial refund (7 Oct 2025); Fortune, Cursor support AI (Apr 2025)
- NIST, AI 600-1 Generative AI Profile; Gibson Dunn, EU AI Act Omnibus agreement (May 2026); RBI, FREE-AI framework (13 Aug 2025)