Research map / Large language models / Retrieval and knowledge
Provence: efficient and robust context pruning for retrieval-augmented generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Provence trains a lightweight DeBERTa-v3 sequence-labeling model on LLM-generated silver labels to prune retrieved contexts sentence by sentence, and unifies pruning with reranking in one forward pass.
Problem
RAG contexts are long and contain irrelevant information; existing context pruners are either token-level, need a fixed compression ratio, are slow LLM-based or autoregressive, or are not robust across domains.
Approach
Cast query-dependent sentence-level context pruning as binary sequence labeling with a small DeBERTa encoder that adaptively keeps zero to all sentences, and merge the pruning head into a cross-encoder reranker so pruning costs almost nothing.
Key results
- Provence (w/ reranker) reaches NQ LLM-Eval 72.6 at 76.0% compression vs 71.8 with full context, and comparable pruning to full context on PopQA with 75.8% compression.
- Provence standalone prunes about 49% of context at near-no loss, with 2x inference speedup at batch size 256 for Llama-2-7B.
- Unified Provence reranking nDCG@10 on BEIR average 55.9 vs 55.4 for the DeBERTa-v3 reranker baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Prune contexts with Provence instead of using full context | RAG with Llama-2-7B-chat · PopQA | LLM-Eval: 57.8 -> 59.3 | Helped |
| Prune contexts with Provence instead of using full context | RAG with Llama-2-7B-chat · NQ | LLM-Eval: 71.8 -> 72.4 | Helped |
| Prune contexts with Provence instead of using full context | RAG with Llama-2-7B-chat · HotpotQA | LLM-Eval: 57.0 -> 56.7 | Hurt |
| Prune contexts with Provence instead of using full context | RAG with Llama-2-7B-chat · TyDi QA | LLM-Eval: 73.9 -> 70.5 | Hurt |
| Unify reranking and pruning in Provence | DeBERTa-v3 reranker · NQ | R@5: 83.0 -> 84.4 | Helped |
| Unify reranking and pruning in Provence | DeBERTa-v3 reranker · TREC DL'19 | nDCG@10: 77.4 -> 77.2 | Hurt |
| Increase model size from DeBERTa-base to DeBERTa-large (ablation) | Anchor context pruner · NQ | LLM-Eval: 71.6 -> 72.1 | Helped |
| Train on equal-size MS MARCO data instead of NQ (ablation) | Anchor context pruner · NQ | LLM-Eval: 71.6 -> 72.8 | Helped |
| Use full MS MARCO training data instead of NQ-sized data (ablation) | Anchor context pruner · NQ | LLM-Eval: 71.6 -> 72.3 | Helped |
| Use relevance oracle instead of answer oracle (ablation) | Anchor context pruner · NQ | LLM-Eval: 71.6 -> 72.6 | No clear effect |
| Use straightforward oracle instead of answer oracle (ablation) | Anchor context pruner · NQ | LLM-Eval: 71.6 -> 72.5 | No clear effect |
| Predict labels per sentence instead of per token (ablation) | Anchor context pruner · NQ | LLM-Eval: 71.6 -> 71.0 | Hurt |
Surprising findings
- Pruning sometimes improves downstream QA, e.g., on PopQA and RGB, by filtering noise.
- Threshold-based compression ratio transfers across datasets (about 50-80% pruned) without per-dataset tuning.
Limitations
- Focuses only on QA with a single passage processed at a time.
- Trained on English-only data.
- Lower recall on leftmost and rightmost needle positions due to sparse training examples.
Experimental setup
Datasets: MS MARCO Natural Questions Wikipedia 2024 dump PubMed (ragged) SyllabusQA RGB BEIR TREC DL'19
Benchmarks: Natural Questions HotpotQA TyDi QA PopQA BioASQ SyllabusQA RGB MS MARCO TREC DL'19 BEIR
Metrics: LLM-based evaluation accuracy Recall compression rate MRR@10 nDCG@10 MFLOPS time
Baselines: RECOMP extractive RECOMP abstractive LLMLingua LLMLingua2 LongLLMLingua FilCo COMPACT DSLR
Builds on: RECOMP LLMLingua2 LongLLMLingua DSLR SPLADE-v3
Closest papers by meaning
- RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs (NeurIPS 2024 · similarity 0.89)
- Making Retrieval-Augmented Language Models Robust to Irrelevant Context (ICLR 2024 · similarity 0.89)
- Sufficient Context: A New Lens on Retrieval Augmented Generation Systems (ICLR 2025 · similarity 0.89)
- Retrieval meets Long Context Large Language Models (ICLR 2024 · similarity 0.89)
- Retrieval is Accurate Generation (ICLR 2024 · similarity 0.88)
- Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.