Research map / Large language models / Alignment and preferences
SelfCite: Self-Supervised Alignment for Context Attribution in Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SelfCite, a self-supervised reward from context ablation (necessity and sufficiency via log-prob changes) used for best-of-N citation reranking and SimPO preference optimization to improve sentence-level citations of LLMs.
Problem
LLM citation generation for long-context QA relies on costly human or proprietary-API annotations; the paper seeks annotation-free improvement of citation quality.
Approach
Uses the LLM's own output probabilities under context ablation as a reward: the drop in probability when cited sentences are removed (necessity) plus the retained probability when only cited sentences are kept (sufficiency). This reward drives best-of-N reranking of citation candidates and SimPO preference pairs.
Key results
- Best-of-N with the SelfCite reward raises LongCite-8B average citation F1 from 73.8 to 77.5 on LongBench-Cite.
- SimPO on 2K self-generated preference pairs reaches 77.9 F1, and applying BoN on top gives 79.1 F1 (+5.3 over the LongCite-8B reproduction).
- In the fully self-supervised setting, SFT on ContextCite data plus SimPO and BoN reaches 71.5 F1 versus 65.7 for SFT alone.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply BoN citation reranking with the SelfCite reward | LongCite-8B · LongBench-Cite | citation F1: 73.8 -> 77.5 | Helped |
| Fine-tune with SimPO using SelfCite preference data | LongCite-8B · LongBench-Cite | citation F1: 73.8 -> 77.9 | Helped |
| Apply BoN after SimPO fine-tuning | LongCite-8B · LongBench-Cite | citation F1: 73.8 -> 79.1 | Helped |
| Replace the SelfCite reward with NLI-based citation rewards in SimPO (ablation) | LongCite-8B · LongBench-Cite | citation F1: 72.0 -> 75.0 | Helped |
| Use SimPO after ContextCite-generated SFT | Llama-3.1-8B-Instruct · LongBench-Cite | citation F1: 65.7 -> 69.9 | Helped |
| Apply BoN after SimPO in the fully self-supervised setting | Llama-3.1-8B-Instruct · LongBench-Cite | citation F1: 69.9 -> 71.5 | Helped |
| Use LM log probability rather than the SelfCite reward to rank BoN candidates (ablation) | LongCite-8B · HotpotQA | citation F1: 64.1 -> 63.4 | Hurt |
| Select BoN candidates with the longest citations (ablation) | LongCite-8B · HotpotQA | citation precision: 77.9 -> 73.6 | Hurt |
| Use Prob-Drop alone to rank BoN candidates (ablation) | LongCite-8B · HotpotQA | citation F1: 64.1 -> 66.6 | Helped |
| Use Prob-Hold alone to rank BoN candidates (ablation) | LongCite-8B · HotpotQA | citation F1: 64.1 -> 67.0 | Helped |
| Lower the BoN citation length cap to 256 tokens (ablation) | LongCite-8B · HotpotQA | citation F1: 68.8 -> 66.4 | Hurt |
| Raise the BoN citation length cap to 512 tokens (ablation) | LongCite-8B · HotpotQA | citation F1: 68.8 -> 68.5 | No clear effect |
Surprising findings
- Off-policy denoising of randomly shifted citations degrades performance rather than helping.
- SimPO training with 8K examples performs worse than 2K, with citation length rising to 158.1 tokens, attributed to off-policy drift.
Limitations
- Assumes access to model output probabilities, which closed-source models may not provide.
- Only a sentence-level corroborative benchmark is used; PPO and other RL algorithms are left unexplored.
- Improves citations for already-structured outputs; cannot yet kick-start citation generation from scratch without SFT data.
Experimental setup
Datasets: LongCite-45K LongBench-Cite ALCE
Benchmarks: LongBench-Cite ALCE
Metrics: citation recall citation precision citation F1 citation length answer correctness
Baselines: LongCite-8B LongCite-9B GPT-4o Claude-3-sonnet GLM-4 ContextCite SimPO w/ NLI Rewards Claude Citations
Builds on: SimPO ContextCite LongCite DPO
Closest papers by meaning
- Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection (ICLR 2024 · similarity 0.84)
- Chain-of-Thought Improves Text Generation with Citations in Large Language Models (AAAI 2024 · similarity 0.84)
- ContextCite: Attributing Model Generation to Context (NeurIPS 2024 · similarity 0.83)
- SCOPE: A Self-supervised Framework for Improving Faithfulness in Conditional Text Generation (ICLR 2025 · similarity 0.83)
- Retrieval is Not Enough: Enhancing RAG through Test-Time Critique and Optimization (NeurIPS 2025 · similarity 0.83)
- HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.