Research map / Large language models / Language, safety and interpretability
Precise Information Control in Long-Form Text Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Precise Information Control (PIC), a claim-level task and benchmark (PIC-Bench) for faithfulness hallucination in long-form generation, and PIC-LM, an 8B model trained with SFT plus length-normalized DPO on weakly supervised preference pairs.
Problem
Language models often add unsupported claims or omit required input information when generating long-form text grounded in given claims, i.e., faithfulness hallucination.
Approach
Frame faithfulness as claim-level control with full (include all input claims) and partial (include a subset, no additions) settings, and build preference pairs by perturbing claim sets, selecting chosen/rejected pairs via a normalized log-probability drop from a reference model instead of an API judge.
Key results
- PIC-LM 8B raises average full-setting F1 from 69.1% (Llama 3.1 8B Instruct) to 91.0%, slightly above GPT-4o at 90.5%.
- PIC-LM 8B reaches 93.3% average precision in the partial setting, versus 90.2% for GPT-4o and 73.6% for Llama 3.1 8B Instruct.
- On ASQA, PIC-LM 8B reaches 61.5% standard EM versus 52.5% for Llama 3.1 8B Instruct and 56.2% for CAD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add PIC-formatted SFT and preference tuning | Llama 3.1 8B Instruct · PIC-Bench full (average) | F1: 69.1 -> 91.0 | Helped |
| Add PIC-formatted SFT and preference tuning | Llama 3.1 8B Instruct · PIC-Bench partial (average) | precision: 73.6 -> 93.3 | Helped |
| Add preference optimization after PIC-formatted SFT (ablation) | PIC-LM · PIC-Bench full (average) | F1: 81.9 -> 91.0 | Helped |
| Add preference optimization after PIC-formatted SFT (ablation) | PIC-LM · PIC-Bench partial (average) | precision: 87.6 -> 93.3 | Helped |
| Add PIC-LM as final generator in retrieval-augmented generation | ASQA standard, compared with Llama 3.1 8B Instruct · ASQA standard | exact match: 52.5 -> 61.5 | Helped |
| Add PIC-LM as final generator in retrieval-augmented generation | ASQA oracle, compared with Llama 3.1 8B Instruct · ASQA oracle | exact match: 56.2 -> 65.9 | Helped |
| Use PIC-LM for final generation in the self-verification pipeline | Llama 3.3 70B Instruct pipeline · Birthplace | factual precision: 65.9 -> 86.0 | Helped |
| Use PIC-LM for final generation in the self-verification pipeline | Llama 3.3 70B Instruct pipeline · QAMParI | F1@5: 13.5 -> 22.6 | Helped |
| Add PIC-formatted SFT and preference tuning | Llama 3.1 8B Instruct · Instruction following (average across PIC-Bench tasks) | Prometheus score: 3.75 -> 3.92 | Helped |
| Replace GPT-4o mini evaluator with GPT-4.1 (ablation) | PIC-LM evaluation · PIC-Bench full (average) | F1: 89.9 -> 87.5 | Hurt |
| Replace GPT-4o mini evaluator with GPT-4.1 (ablation) | PIC-LM evaluation · PIC-Bench partial (average) | precision: 91.2 -> 88.1 | Hurt |
| Replace GPT-4o mini evaluator with Claude 3.5 Sonnet (ablation) | PIC-LM evaluation · PIC-Bench full (average) | F1: 89.9 -> 88.2 | Hurt |
Surprising findings
- Even frontier LLMs produce faithfulness hallucinations in over 70% of full-PIC outputs, and perfect-F1 does not exceed 30% on average.
- Claude 3.5 Sonnet scores only about 1% F1 on PopBios-CF because it frequently abstains from producing counterfactual outputs.
Limitations
- Evaluation sets are small (each under 250 samples) due to the cost of claim decomposition and verification.
- PIC assumes well-formed, non-contradictory input claims and does not evaluate instruction following in the main metric.
- PIC-LM is more prone to emitting counterfactual or harmful content when given certain contexts, and its standalone utility is limited.
Experimental setup
Datasets: No Robots FLAN CNN/DailyMail EntityBios PopBios ELI5 AskHistorians ExpertQA FACTS Grounding XSUM ASQA QAMParI
Benchmarks: PIC-Bench ASQA QAMParI Birthplace factoid
Metrics: F1 precision perfect F1 perfect precision exact match recall factual precision F1@5 recall@5 Prometheus score
Baselines: Llama 3.1 8B Instruct Llama 3.3 70B Instruct Tulu 3 8B Ministral 8B Instruct Hermes 3 8B Qwen 3 32B QwQ 32B R1-Qwen-32B GPT-4o Claude 3.5 Sonnet CAD SelfCite 8B
Builds on: DPO Alpaca FActScore Chain-of-Verification Context-aware decoding
Closest papers by meaning
- SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generation (ICLR 2025 · similarity 0.89)
- LongGenBench: Benchmarking Long-Form Generation in Long Context LLMs (ICLR 2025 · similarity 0.89)
- Attention Satisfies: A Constraint-Satisfaction Lens on Factual Errors of Language Models (ICLR 2024 · similarity 0.88)
- LongWriter: Unleashing 10,000+ Word Generation from Long Context LLMs (ICLR 2025 · similarity 0.88)
- Towards Verifiable Text Generation with Generative Agent (AAAI 2025 · similarity 0.88)
- ParamMute: Suppressing Knowledge-Critical FFNs for Faithful Retrieval-Augmented Generation (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.