Research map / Large language models / Retrieval and knowledge
Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies self-contradictions in instruction-tuned LM open-domain generation and proposes a prompting-only pipeline (trigger, detect, iteratively mitigate) that works on black-box models without retrieval.
Problem
Instruction-tuned LMs produce hallucinations in the form of self-contradictory sentence pairs within the same context; there is no black-box method to detect and mitigate these without external retrieval or many samples.
Approach
Self-contradiction reveals non-factuality because two contradicting sentences cannot both be correct; the framework triggers contradictions by generating a second sentence under a constrained context (entity, prefix, partial relation triple), detects them with chain-of-thought prompting, and iteratively revises sentences locally to remove the conflicting information.
Key results
- ChatGPT produced self-contradictions in 17.7% of sentences in open-domain generation, with 35.2% of these unverifiable with online text.
- Using ChatGPT as the analyzer, detection reached 84.2% precision, 83.2% recall, and 83.7% F1 on MainTestSet; mitigation removed 89.5% of self-contradictions while retaining 100.8% of informative facts.
- On PopQA question answering, ChatGPT-as-analyzer detected 12.7% to 38.0% self-contradictions at 74.2% to 83.8% precision across vanilla and retrieval-augmented settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed context-constrained trigger prompt instead of Continue, Rephrase, or Q&A (ablation) | ChatGPT sentence-pair generation on MainTestSet · MainTestSet (ChatGPT-generated text) | Self-contradictions triggered: 5.1%, 0.2%, and 10.3% -> 17.7% | Helped |
| Use the proposed context-constrained trigger prompt instead of Continue, Rephrase, or Q&A (ablation) | ChatGPT sentence-pair generation on MainTestSet · MainTestSet (ChatGPT-generated text) | Detection precision: 76.8%, 33.3%, and 81.9% -> 84.2% | Helped |
| Use two-turn chain-of-thought detection instead of Directly ask or Step-by-step (ablation) | ChatGPT detector · MainTestSet | Detection F1: 73.1%, 67.4% -> 83.7% | Helped |
| Use multi-path reasoning instead of the proposed detection prompt (ablation) | ChatGPT detector · MainTestSet | Detection F1: 83.7% -> 82.0% | Hurt |
| Iterate mitigation and remove remaining predicted-contradictory sentences at the end, rather than naively dropping all at the beginning (ablation) | ChatGPT mitigation · MainTestSet (ChatGPT-generated text) | Informative facts retained: 89.4% -> 100.8% | Helped |
| Iterate mitigation and remove remaining predicted-contradictory sentences at the end, rather than naively dropping all at the beginning (ablation) | ChatGPT mitigation · MainTestSet (ChatGPT-generated text) | Perplexity increased: 0.72 -> 0.44 | Helped |
| Use GPT-4 rather than ChatGPT as the analyzer model | Detector for ChatGPT-generated sentence pairs · MainTestSet | Detection F1: 83.7% -> 86.5% | Helped |
| Use Llama2-70B-Chat rather than ChatGPT as the analyzer model | Detector for ChatGPT-generated sentence pairs · MainTestSet | Detection F1: 83.7% -> 61.9% | Hurt |
| Use Vicuna-13B rather than ChatGPT as the analyzer model | Detector for ChatGPT-generated sentence pairs · MainTestSet | Detection F1: 83.7% -> 39.3% | Hurt |
| Use retrieval augmentation for question answering instead of vanilla generation | ChatGPT question answering with ChatProtect detection · PopQA | Self-contradictions predicted: 18.2% -> 12.7%; 83.2% -> 83.8% precision | Mixed |
| Vary aLM detection temperature from 0 to 1 (ablation) | ChatGPT detector · MainTestSet | Detection F1: 84.0%, 83.9%, 83.6%, 81.8%, 83.7% | No clear effect |
| Vary gLM text-generation temperature from 0 to 1 (ablation) | ChatGPT trigger and detector · MainTestSet | Detection precision: 84.1%, 81.3%, 79.2%, 83.3%, 83.4% | No clear effect |
Surprising findings
- Rephrase-based triggering performed poorly (0.2% triggered, 33.3% precision), while the more constrained prompt triggered more self-contradictions.
- Step-by-step reasoning had lower F1 than directly asking or multi-path reasoning in the detection ablation, contrary to expectations, while chain-of-thought with two turns performed best.
Limitations
- Only handles contradictions between two sentences sampled at the same text position
- Open-source analyzers (Llama2-70B-Chat, Vicuna-13B) have low detection recall
- Post-processing approach; does not change the generator to avoid contradictions
Experimental setup
Datasets: WikiBio PopQA
Benchmarks: MainTestSet 2ndTestSet PopQA
Metrics: precision recall F1 self-contradictions removed informative facts retained perplexity increase
Baselines: SelfCheckGPT (unigram, BERTScore, MQAG) Directly ask Step-by-step reasoning Multi-path reasoning Continue Rephrase Q&A Naively drop
Builds on: chain-of-thought prompting CompactIE SelfCheckGPT
Closest papers by meaning
- Large Language Models Cannot Self-Correct Reasoning Yet (ICLR 2024 · similarity 0.88)
- FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees (ICML 2025 · similarity 0.88)
- How Language Model Hallucinations Can Snowball (ICML 2024 · similarity 0.87)
- Small Language Model Can Self-Correct (AAAI 2024 · similarity 0.87)
- When LLMs Meet Cunning Texts: A Fallacy Understanding Benchmark for Large Language Models (NeurIPS 2024 · similarity 0.87)
- Teaching Language Models to Hallucinate Less with Synthetic Tasks (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.