Research map / Large language models / Instruction and fine-tuning
Fine-Tuning Language Models for Factuality
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes Llama models for factuality in long-form generation using DPO on automatically constructed preference pairs, scored either by FactScore (reference-based) or by model-confidence over resampled answers (reference-free), without human labels.
Problem
Language models produce confident but factually incorrect long-form claims, and human factuality labeling is too expensive to use as a training signal for reducing hallucinations.
Approach
Build factuality preference pairs without human labels by scoring each response's atomic claims either with FactScore against Wikipedia or with the model's own sampled-answer consistency, then optimize the policy with DPO; the reference-free variant needs no retrieval.
Key results
- At 7B scale, FactTune-FS on Llama-2 raises biography % correct to 0.895 vs 0.748 for Llama-2-Chat (about 58% reduction in factual error rate, per abstract).
- On Medical QA, FactTune-FS on Llama-2 reaches 0.783 % correct vs 0.636 for SFT-free Llama-2 Chat baseline (about 40% error reduction vs chat).
- FactTune-FS on Llama-2-Chat improves biography % correct from 0.748 to 0.831, and DoLa decoding adds further gains on Llama-1 (biographies 0.812 to 0.864).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune on reference-based FactScore preferences instead of SFT | Llama-1 and Llama-2 · Biographies and Medical QA | % Correct: 0.568 -> 0.812; 0.630 -> 0.707; 0.701 -> 0.895; 0.635 -> 0.783 | Helped |
| Fine-tune on model-confidence preferences instead of SFT | Llama-1 and Llama-2 · Biographies and Medical QA | % Correct: 0.568 -> 0.783; 0.630 -> 0.642; 0.701 -> 0.846; 0.635 -> 0.704 | Helped |
| Fine-tune Llama-2-Chat with FactTune-FS preferences | Llama-2-Chat · Biographies and Medical QA | % Correct: 0.748 -> 0.831; 0.636 -> 0.682 | Helped |
| Fine-tune Llama-2-Chat with FactTune-MC preferences | Llama-2-Chat · Biographies and Medical QA | % Correct: 0.748 -> 0.812; 0.636 -> 0.645 | Helped |
| Add DOLA decoding to FactTune-FS | Llama-1 and Llama-2 · Biographies and Medical QA | %Correct: 0.812 -> 0.864; 0.707 -> 0.767; 0.895 -> 0.865; 0.783 -> 0.794 | Mixed |
| Replace entity or noun-chunk extraction with atomic question extraction (ablation) | FactTune-MC scoring · Biographies and Medical QA | %Correct: 0.693 -> 0.810; 0.660 -> 0.711 | Helped |
| Use maximum confidence instead of entropy (ablation) | Atomic-extraction FactTune-MC scoring with heuristic matching · Biographies | %Correct: 0.810 -> 0.840 | Helped |
| Use maximum confidence instead of entropy (ablation) | Atomic-extraction FactTune-MC scoring with heuristic matching · Medical QA | %Correct: 0.711 -> 0.673 | Hurt |
| Replace heuristic equivalence matching with GPT-3.5 matching (ablation) | Atomic-extraction FactTune-MC scoring · Biographies and Medical QA | %Correct: 0.840 -> 0.794; 0.673 -> 0.668 | Hurt |
Surprising findings
- Using GPT-3.5 for answer equivalence checking yielded worse preference pairs than a simple string-matching heuristic.
- Factuality-tuned models produce shorter, terser responses, and DoLa decoding does not help factuality-tuned Llama-2 on biographies.
Limitations
- Only 7B models studied; scaling behavior unknown.
- Reference-based method requires reliable reference texts (Wikipedia) and fails in niche domains.
- Evaluation limited to two tasks (biographies and medical QA) that may not cover the full range of factuality scenarios.
Experimental setup
Datasets: Biographies dataset generated with GPT-3.5 prompts and Llama-1-7B responses Medical QA dataset generated with GPT-3.5 prompts and Llama-1-7B responses Wikipedia
Benchmarks: Biographies (FactScore-style evaluation) Medical QA (50 test conditions)
Metrics: # correct facts # incorrect facts % correct facts human-rated accuracy GPT-4 error count
Baselines: Llama-2-Chat ITI DoLa SFT
Builds on: Direct Preference Optimization FactScore Inference-time intervention DoLa
Closest papers by meaning
- Long-form factuality in large language models (NeurIPS 2024 · similarity 0.93)
- Is Factuality Enhancement a Free Lunch For LLMs? Better Factuality Can Lead to Worse Context-Faithfulness (ICLR 2025 · similarity 0.89)
- FactTest: Factuality Testing in Large Language Models with Finite-Sample and Distribution-Free Guarantees (ICML 2025 · similarity 0.89)
- The FACTS Leaderboard: A Comprehensive Benchmark for Large Language Model Factuality (NeurIPS 2026 · similarity 0.89)
- DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models (ICLR 2024 · similarity 0.89)
- Understanding Finetuning for Factual Knowledge Extraction (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.