Research map / Large language models / Retrieval and knowledge
Influence Guided Context Selection for Effective Retrieval-Augmented Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Contextual Influence (CI) value, an inference-time data-valuation metric for RAG context selection, and a hierarchical surrogate model (CSM) trained to predict it, keeping contexts with positive CI values.
Problem
Retrieved contexts in RAG often contain irrelevant or noisy information that degrades generator output, and existing context selection metrics (query-aware, list-aware, generator-aware) give limited gains over standard RAG and need top-k tuning.
Approach
Reframe context quality as leave-one-out influence (CI value = utility drop when a context is removed), which jointly captures query relevance, list uniqueness and generator alignment; approximate it at inference with a BERT-based hierarchical surrogate (local query-context encoder plus global self-attention) trained by oracle CI regression with imbalance handling and/or end-to-end soft masking th
Key results
- CSM-st reaches 42.53 EM on NQ with Llama3-8B vs 37.01 for Standard RAG and 39.06 for bge-reranker (top-k=5).
- Authors report CSM outperforms all baselines by 21.72% in EM on open-domain QA and 19.40% F1 on multihop QA, with 15.03% average improvement over leading baselines.
- CSM predictions reach Spearman correlation above 0.75 with oracle CI values across tasks for both backbones.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use supervised CI surrogate context selection (CSM-st) instead of the strongest listed baseline | Llama3-8B RAG · NQ | EM: 41.77 -> 42.53 | Helped |
| Use supervised CI surrogate context selection (CSM-st) instead of the strongest listed baseline | Llama3-8B RAG · TriviaQA | EM: 65.83 -> 69.59 | Helped |
| Use supervised CI surrogate context selection (CSM-st) instead of the strongest listed baseline | Llama3-8B RAG · WebQA | EM: 26.12 -> 24.77 | Hurt |
| Use supervised CI surrogate context selection (CSM-st) instead of the strongest listed baseline | Llama3-8B RAG · HotpotQA | F1: 45.69 -> 47.53 | Helped |
| Use supervised CI surrogate context selection (CSM-st) instead of the strongest listed baseline | Qwen2.5-7B RAG · NQ | EM: 45.72 -> 47.38 | Helped |
| Use supervised CI surrogate context selection (CSM-st) instead of the strongest listed baseline | Qwen2.5-7B RAG · 2Wiki | F1: 43.27 -> 48.67 | Helped |
| Use supervised CI surrogate context selection (CSM-st) instead of the strongest listed baseline | Qwen2.5-7B RAG · TruthfulQA | Accuracy: 32.07 -> 28.77 | Hurt |
| remove data intervention (ablation) | CSM-st · NQ | EM: 42.53 -> 37.67 | Hurt |
| add data intervention (ablation) | CSM-st · TriviaQA | EM: 62.91 -> 69.59 | Helped |
| add data intervention (ablation) | CSM-st · WebQA | EM: 19.23 -> 24.77 | Helped |
| remove contrastive loss (ablation) | CSM-st · NQ | EM: 42.53 -> 40.82 | Hurt |
| remove sufficiency loss (ablation) | CSM-e2e · NQ | EM: 41.61 -> 35.44 | Hurt |
Surprising findings
- Several existing selection metrics (e.g., RankGPT, bge-reranker) sometimes underperform or barely beat standard RAG, and RECOMP-ex falls below standard RAG on some tasks (e.g., 1.7% lower on ASQA).
- About 78% of NQ contexts have near-zero CI values, so most retrieved contexts have little influence on generation.
Limitations
- CSM training remains challenging and requires task-specific optimization; no universal selector across tasks.
- Oracle CI values require n+1 generator forward passes per training query, and the approach is evaluated only on 8 tasks with 2 generators.
Experimental setup
Datasets: NQ TriviaQA WebQA HotpotQA 2WikiMultiHopQA FEVER TruthfulQA ASQA Wikipedia Dec 2018 dump
Benchmarks: NQ TriviaQA WebQA HotpotQA 2WikiMultiHopQA FEVER TruthfulQA ASQA
Metrics: Exact Match F1 Accuracy Spearman correlation inference latency (ms)
Baselines: Standard RAG bge-reranker RankGPT RECOMP-ex RECOMP-abs Ret-Robust Self-RAG RQ-RAG
Builds on: RECOMP FlashRAG RankGPT Ret-Robust leave-one-out data valuation
Closest papers by meaning
- Inference Scaling for Long-Context Retrieval Augmented Generation (ICLR 2025 · similarity 0.91)
- Sufficient Context: A New Lens on Retrieval Augmented Generation Systems (ICLR 2025 · similarity 0.90)
- Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG (ICLR 2025 · similarity 0.90)
- Accelerating Inference of Retrieval-Augmented Generation via Sparse Context Selection (ICLR 2025 · similarity 0.88)
- LaRA: Benchmarking Retrieval-Augmented Generation and Long-Context LLMs – No Silver Bullet for LC or RAG Routing (ICML 2025 · similarity 0.88)
- Reliable Decision‑Making via Calibration‑Oriented Retrieval‑Augmented Generation (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.