Research map / Large language models / Language, safety and interpretability
ANAH-v2: Scaling Analytical Hallucination Annotation of Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ANAH-v2, an EM-based iterative self-training framework that scales a sentence-level hallucination annotation dataset and a 7B annotator (InternLM2-7B) over three data-scaling stages, with self-consistency in the E-step.
Problem
Fine-grained hallucination annotation datasets are small and costly to create by humans or GPT-4, and existing automatic annotators are not accurate enough to scale oversight of LLM hallucinations.
Approach
Treat dataset scaling and annotator training as an EM loop: the current annotator labels new responses with self-consistency (majority vote on hallucination type, cosine-similarity selection of reference points), then a new annotator is trained on the enlarged data; a phased multi-turn annotation format (factual existence, reference extraction, type judgment) replaces ANAH's single-turn format.
Key results
- ANAH-v2 Stage3 (7B) reaches 89.55% accuracy on the ANAH test set vs. GPT-4's 86.97%.
- Zero-shot accuracy of 81.54% on HaluEval and 94.44% on HalluQA, reported as new SOTA over KnowHalu (80.70%) and GPT-4 (65.05% on HaluEval).
- Re-ranking with the annotator raises the NLI metric for LLaMA2-7B on HaluEval from 31.25 to 36.03 (and InternLM2-7B from 25.00 to 37.01).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Scale training data from Stage 1 to Stage 2 | ANAH-v2 annotator · ANAH test set | Accuracy: 84.85 -> 88.18 | Helped |
| Scale training data from Stage 2 to Stage 3 with additional topics | ANAH-v2 annotator · ANAH test set | Accuracy: 88.18 -> 89.55 | Helped |
| Use ANAH-v2-Stage3 instead of GPT-4 | Hallucination detection · HaluEval, zero-shot | Accuracy: 65.05 -> 81.54 | Helped |
| Use ANAH-v2-Stage3 instead of GPT-4 | Hallucination detection · HalluQA, zero-shot | Accuracy: 62.81 -> 94.44 | Helped |
| Add self-consistency at inference (ablation) | ANAH-v2-Stage1 · ANAH test set | Accuracy: 81.67 -> 84.85 | Helped |
| Add self-consistency to training data while retaining inference self-consistency (ablation) | ANAH-v2-Stage2 · ANAH test set | Accuracy: 84.24 -> 88.18 | Helped |
| Add self-consistency at inference with self-consistent training data (ablation) | ANAH-v2-Stage2 · ANAH test set | Accuracy: 85.15 -> 88.18 | Helped |
| Add progressive data scaling instead of annotating added data with the Stage 1 annotator (ablation) | ANAH-v2-Stage3 · ANAH test set | Accuracy: 86.36 -> 89.55 | Helped |
| Mix newly annotated data with old data rather than further-train on new data only (ablation) | ANAH-v2-Stage2 · ANAH test set | Accuracy: 85.91 -> 88.18 | Helped |
| Mix newly annotated data with old data rather than further-train on new data only (ablation) | ANAH-v2-Stage3 · ANAH test set | Accuracy: 86.52 -> 89.55 | Helped |
| Re-rank generated responses using ANAH-v2 | LLaMA2-7B · HaluEval | NLI: 31.25 -> 36.03 | Helped |
| Re-rank generated responses using ANAH-v2 | InternLM2-7B · HaluEval | NLI: 25.00 -> 37.01 | Helped |
Surprising findings
- GPT-4 has higher RougeL and BERTScore than ANAH-v2, which the authors attribute to GPT-4 pre-annotations in the seed data rather than annotation quality.
- Stage 1 ANAH-v2 (7B) already outperforms ANAH-20B on accuracy (84.85% vs 81.01%) with the same training corpus, attributed to the multi-turn training format.
Limitations
- Annotation margin of error remains and may affect convergence and final annotator quality.
- Only InternLM2-7B backbone explored; other base models and sizes not tested.
- EM is sensitive to initialization; global optimality not guaranteed. Evaluation relies on own dataset and HaluEval/HalluQA, which may not cover real-world scenarios.
Experimental setup
Datasets: ANAH HaluEval HalluQA HotPotQA ShareGPT Dolly
Benchmarks: ANAH HaluEval HalluQA
Metrics: F1 accuracy RougeL BERTScore NLI QuestEval
Baselines: GPT-4 ANAH-7B ANAH-20B KnowHalu WiKiChat HaluEval
Builds on: ANAH self-consistency InternLM2
Closest papers by meaning
- Scalable Token-Level Hallucination Detection in Large Language Models (NeurIPS 2026 · similarity 0.94)
- HaDeMiF: Hallucination Detection and Mitigation in Large Language Models (ICLR 2025 · similarity 0.93)
- LLM-Check: Investigating Detection of Hallucinations in Large Language Models (NeurIPS 2024 · similarity 0.93)
- Detecting and Mitigating Hallucination in Large Vision Language Models via Fine-Grained AI Feedback (AAAI 2025 · similarity 0.92)
- FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees (ICML 2025 · similarity 0.91)
- Attributive Reasoning for Hallucination Diagnosis of Large Language Models (AAAI 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.