Research map / Large language models / Instruction and fine-tuning
Improving Factuality in Large Language Models via Decoding-Time Hallucinatory and Truthful Comparators
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CDT, a decoding-time framework that contrasts logits of a target LLM against LoRA-trained hallucinatory and truthful comparators, with instruction-prototype-guided mixture of experts to handle multi-task hallucination patterns.
Problem
LLMs generate content contradicting verifiable facts, and existing fixes (parameter optimization, representation editing) can compromise internal knowledge and fail to handle multifaceted hallucination patterns across tasks.
Approach
Build hallucinatory and truthful comparators via LoRA SFT on explicit hallucinated/factual pairs, route them with instruction-prototype (GMM-based) soft gated mixture of experts, and combine their logit differences with the target model at decoding time under an adaptive plausibility constraint.
Key results
- On TruthfulQA, CDT reaches 50.74% MC1, 75.96% MC2 and 50.97% MC3, outperforming ICD (46.32/69.08/41.25) with the best scores on all metrics.
- On KNIGHT-Judge, CDT achieves 65.77% F1 versus 63.93% for ICD and 51.58% for the baseline.
- On XSUM, CDT improves FactKB from 38.79 to 73.66 and BERTScore from 82.50 to 86.44 over the LLaMA2-7B-Chat baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the hallucinatory comparator (ablation) | CDT · KNIGHT-Judge | F1 Score: 62.05 -> 65.77 | Helped |
| Replace the base hallucinatory comparator with its chat version (ablation) | CDT · KNIGHT-Judge | F1 Score: 65.77 -> 64.03 | Hurt |
| Add the truthful comparator (ablation) | CDT · KNIGHT-Judge | F1 Score: 54.61 -> 65.77 | Helped |
| Replace the base truthful comparator with its chat version (ablation) | CDT · KNIGHT-Judge | F1 Score: 65.77 -> 65.75 | No clear effect |
| Add PME instead of using a single LoRA (ablation) | CDT · KNIGHT-Judge | F1 Score: 62.94 -> 65.77 | Helped |
| Add instruction prototypes (ablation) | CDT · KNIGHT-Judge | F1 Score: 64.06 -> 65.77 | Helped |
| Add perturbation adversarial training (PAT) (ablation) | CDT · KNIGHT-Judge | F1 Score: 64.65 -> 65.77 | Helped |
| Replace PAT with PGD-based adversarial training (ablation) | CDT · KNIGHT-Judge | F1 Score: 65.18 -> 65.77 | Hurt |
| Include QA data in comparator SFT (ablation) | CDT · TruthfulQA | MC1: 48.12 -> 50.74 | Helped |
| Include dialogue data in comparator SFT (ablation) | CDT · TruthfulQA | MC1: 48.93 -> 50.74 | Helped |
| Include summarization data in comparator SFT (ablation) | CDT · XSUM | BERTScore: 84.45 -> 86.44 | Helped |
| Include general-task data in comparator SFT (ablation) | CDT · Alpaca-Gen | DAE: 54.24 -> 56.50 | Helped |
Surprising findings
- The hallucinatory comparator from the chat version of the model gave minor/ineffective gains, which the authors attribute to RLHF already encoding honesty.
- Removing the truthful comparator lowered KNIGHT-Judge F1 sharply (65.77 to 54.61), more than removing the hallucinatory comparator (62.05).
Limitations
- Requires two additional forward passes at inference, increasing latency.
- Cannot create knowledge beyond the target model's internal knowledge.
- Relies on GPT-4-generated hallucinated and expanded factual responses with manual checks.
Experimental setup
Datasets: HaluEval Alpaca OpenDialKG TruthfulQA XSUM
Benchmarks: KNIGHT-Judge Alpaca-Judge TruthfulQA XSUM KNIGHT-Gen Alpaca-Gen
Metrics: Accuracy Precision Recall F1 Score MC1 MC2 MC3 ROUGE-1 ROUGE-2 ROUGE-L
Baselines: SFT FPO CD ITI SH2 DoLa ICD
Builds on: Contrastive Decoding DoLa ICD ITI DPO
Closest papers by meaning
- DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models (ICLR 2024 · similarity 0.92)
- In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation (ICML 2024 · similarity 0.91)
- FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees (ICML 2025 · similarity 0.91)
- FLAME : Factuality-Aware Alignment for Large Language Models (NeurIPS 2024 · similarity 0.90)
- MoLE:Decoding by Mixture of Layer Experts Alleviates Hallucination in Large Vision-Language Models (AAAI 2025 · similarity 0.90)
- OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.