Research map / Large language models / Language, safety and interpretability
ContextCite: Attributing Model Generation to Context
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces context attribution (finding which context sources cause a generated statement) and ContextCite, a sparse linear surrogate built from random context ablations, and applies it to verification, pruning, and poisoning detection.
Problem
Identifying which parts of a provided context actually led a language model to generate a given statement, distinguishing grounded, misinterpreted, and fabricated content.
Approach
Treat the LM's response probability under random context ablations as a function to learn; fit a sparse (Lasso) linear surrogate on logit-probabilities and use its weights as contributive attribution scores, applicable to any existing LM without training.
Key results
- ContextCite with 32 ablations consistently matches or outperforms attention, gradient norm, semantic similarity and leave-one-out baselines across benchmarks and models on top-k drop and LDS.
- Selecting top-k ContextCite sources improves Llama-3-8B F1 on query-relevant pruning, e.g., on Natural Questions replacing full context with top-8 to 128 sources boosts F1.
- Prompt injection is surfaced as the single most influential source in the NeuralExec attack case with 98.9% top-1 detection accuracy on Llama-3-8B; handcrafted attacks reach 90% top-1 on Phi-3-mini.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ContextCite-attributed sources instead of the entire context for answer verification | Llama-3-8B · HotpotQA and Natural Questions | AUC for predicting answer accuracy | Helped |
| Regenerate answers using only the top-k ContextCite-attributed sources | Llama-3-8B · HotpotQA and Natural Questions | F1-score | Helped |
| Use 32 context ablations to fit the ContextCite surrogate | ContextCite · TyDi QA, Hotpot QA, and CNN DailyMail | top-k log-probability drop and LDS | Helped |
| Increase ContextCite ablations from 32 to 64, 128, or 256 (ablation) | ContextCite · TyDi QA, Hotpot QA, and CNN DailyMail | attribution quality | Helped |
| Use attention scores instead of ContextCite | context attribution · Benchmarks evaluated with Llama-3-8B and Phi-3-mini | top-k log-probability drop and LDS | Mixed |
Surprising findings
- Attention baseline approaches ContextCite on Llama-3-8B but performs poorly on Phi-3-mini, suggesting attention is not consistently reliable for attribution.
- Relevant sources for a generated statement are typically few even in long contexts, enabling sparse surrogates with only 32 ablations.
Limitations
- Attributions are contributive only and do not establish corroboration or support of statements.
- Poisoning detection fails when attacks cause only subtle changes to output, and GCG attacks had low success rate (2%).
- Requires many forward passes per response, and relies on ablation-based removal of sentences.
Experimental setup
Datasets: TyDi QA HotpotQA CNN DailyMail Natural Questions MS MARCO NeuralExec test set
Benchmarks: TyDi QA HotpotQA CNN DailyMail Natural Questions MS MARCO
Metrics: top-k log-probability drop linear datamodeling score RMSE AUC F1 top-1 detection accuracy
Baselines: Leave-one-out Attention Gradient norm Semantic similarity Attention rollout Gradient times input
Builds on: linear datamodeling score TRAK Lasso attention rollout leave-one-out
Closest papers by meaning
- AttriBoT: A Bag of Tricks for Efficiently Approximating Leave-One-Out Context Attribution (ICLR 2025 · similarity 0.87)
- Quantifying the Plausibility of Context Reliance in Neural Machine Translation (ICLR 2024 · similarity 0.84)
- CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models (ICML 2025 · similarity 0.84)
- Context Steering: Controllable Personalization at Inference Time (ICLR 2025 · similarity 0.84)
- LLM Attributor: Interactive Visual Attribution for LLM Generation (AAAI 2025 · similarity 0.84)
- PICASO: Permutation-Invariant Context Composition with State Space Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.