CalibratedAgentsBy Calibrated Agents Research Team8 min read
Benchmark · 10 Oct 2026
Microsoft-Decision-1 vs OpenAI vs Jev vs Cloudflare Clef: decision models benchmarked (2026)
Microsoft launched Microsoft-Decision-1 on 9 October 2026.
Read more →Microsoft-Decision-1 vs OpenAI vs Jev vs Cloudflare Clef: decision models benchmarked (2026)
Microsoft launched Microsoft-Decision-1 on 9 October 2026.Read more → By Calibrated Agents Research Team9 min readHallucination detection in 2026: find the wrong sentence, not just the wrong answer
An AI answer is rarely all wrong.Read more → By Calibrated Agents Research Team8 min readHallucination detection benchmark 2026: 50 labelled answers beat a tuned LLM judge
Most hallucination detection benchmarks compare detectors with whatever thresholds their authors chose.Read more → By Calibrated Agents Research Team5 min readWhy LLM-as-a-judge fails in production: the base rate problem
LLM-as-a-judge has good recall and no precision control.Read more →How to check a RAG answer against its sources with one API call
A step-by-step guide from an API key to your first check.Engineering notes from building Ground
How the checks run, and what we learned shipping them.Nothing matches yet. Try “benchmark”, “judge” or “RAGTruth”.
Product launches
Reading list
The papers and cases our work builds on.
- 01RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Models
Niu et al., 2024 - 02MiniCheck: Efficient Fact-Checking of LLMs on Grounding Documents (LLM-AggreFact)
Tang, Laban & Durrett, 2024 - 03FaithBench: A Diverse Hallucination Benchmark for Summarization by Modern LLMs
Bao et al., NAACL 2025 - 04Lynx: An Open Source Hallucination Evaluation Model (HaluBench)
Ravi et al., Patronus AI, 2024 - 05Hallucination Leaderboard (HHEM)
Vectara, ongoing - 06Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena
Zheng et al., NeurIPS 2023 - 07Self-Consistency Improves Chain of Thought Reasoning in Language Models
Wang et al., ICLR 2023 - 08On Calibration of Modern Neural Networks
Guo et al., ICML 2017 - 09Moffatt v. Air Canada, 2024 BCCRT 149
Civil Resolution Tribunal, British Columbia, 2024
Newsletter
The receipt
One email a month: what we measured, with the data. No promotions.