Research map / Large language models / Retrieval and knowledge
DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DoLa, a training-free decoding method that contrasts logits from a dynamically selected early layer against the final layer to reduce hallucinations in LLaMA models.
Problem
LLMs hallucinate, generating content that deviates from facts learned during pretraining; existing mitigations rely on retrieval, fine-tuning, or human labels.
Approach
Factual knowledge emerges in higher layers, so the next-token distribution is obtained by contrasting log-probabilities of the final (mature) layer against a premature layer chosen per step by maximum Jensen-Shannon divergence within layer buckets, with an adaptive plausibility constraint. It needs no retrieval or fine-tuning.
Key results
- On TruthfulQA open-ended generation, DoLa improves LLaMA-7B %Truth*Info from 26.9 to 40.8 and LLaMA-65B's MC2 from 46.9 to 64.6 on multiple choice.
- On FACTOR, DoLa outperforms the baseline by 2-4 points across LLaMA sizes (e.g., LLaMA-7B News 58.3 to 62.0).
- On GSM8K, DoLa raises LLaMA-65B accuracy from 51.2 to 54.0 and LLaMA-33B from 33.8 to 35.5, with latency overhead of 1.01-1.08x.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply DoLa decoding | LLaMA-7B · TruthfulQA MC | MC2: 40.6 -> 63.8 | Helped |
| Apply DoLa decoding | LLaMA-7B · FACTOR News | accuracy: 58.3 -> 62.0 | Helped |
| Apply DoLa decoding | LLaMA-7B · TruthfulQA open-ended | %Truth: 30.4 -> 42.1 | Helped |
| Apply DoLa decoding | LLaMA-7B · StrategyQA | accuracy: 60.1 -> 64.1 | Helped |
| Apply DoLa decoding | LLaMA-7B · GSM8K | accuracy: 10.8 -> 10.5 | Hurt |
| Use fixed premature-layer selection rather than dynamic selection (ablation) | DoLa on LLaMA-7B · TruthfulQA MC | MC2: 63.8 -> 68.3 | Helped |
| Replace JSD-based selection with random layer selection (ablation) | DoLa on LLaMA-7B · FACTOR News | accuracy: 58.3 -> 60.0, compared with DoLa 62.0 | Hurt |
| Apply post-softmax to the contrast distribution (ablation) | DoLa on LLaMA-7B · TruthfulQA MC | MC2: 63.8 -> 52.2 | Hurt |
| Contrast only with the word embedding layer (ablation) | DoLa on LLaMA-7B · TruthfulQA MC | MC2: 63.8 -> 61.7 | Hurt |
| Select dynamically from all early even-numbered layers (ablation) | DoLa on LLaMA-7B · TruthfulQA MC | MC2: 63.8 -> 63.9 | No clear effect |
| Apply DoLa decoding (ablation) | GPT2-Medium · TruthfulQA MC | MC1: 23.5 -> 22.9 | Hurt |
| Apply a repetition penalty (ablation) | DoLa · StrategyQA and GSM8K | accuracy | Helped |
Surprising findings
- Contrastive decoding with a LLaMA-7B amateur model reduces GSM8K and StrategyQA accuracy for larger models, and CD frequently outputs 'I have no comment' (over 60% for LLaMA-33B).
- DoLa does not improve GPT2-Medium factuality, suggesting layer contrast requires large models with distinct layer-wise knowledge.
Limitations
- Focuses only on factuality; not explored with RLHF or other dimensions.
- Inference only; relies on existing parameters and cannot correct misinformation learned during pretraining.
- Does not help small models such as GPT2-Medium on multiple-choice factuality.
Experimental setup
Datasets: TruthfulQA FACTOR StrategyQA GSM8K Vicuna QA CoNLL-2003
Benchmarks: TruthfulQA FACTOR StrategyQA GSM8K Vicuna QA
Metrics: MC1 MC2 MC3 %Truth %Info %Truth*Info %Reject accuracy GPT-4 pairwise score
Baselines: Original greedy decoding Contrastive Decoding (CD) Inference Time Intervention (ITI) DoLa-static
Builds on: Contrastive Decoding early exit adaptive plausibility constraint Inference Time Intervention
Closest papers by meaning
- Improving Factuality in Large Language Models via Decoding-Time Hallucinatory and Truthful Comparators (AAAI 2025 · similarity 0.92)
- FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees (ICML 2025 · similarity 0.91)
- In-Context Sharpness as Alerts: An Inner Representation Perspective for Hallucination Mitigation (ICML 2024 · similarity 0.90)
- LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations (ICLR 2025 · similarity 0.90)
- MoLE:Decoding by Mixture of Layer Experts Alleviates Hallucination in Large Vision-Language Models (AAAI 2025 · similarity 0.89)
- SLED: Self Logits Evolution Decoding for Improving Factuality in Large Language Models (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.