Research map / Large language models / Retrieval and knowledge
Beyond RAG vs. Long-Context: Learning Distraction-Aware Retrieval for Efficient Knowledge Grounding
Seong-Woong Shim, Myunsoo Kim, Jae Hyeon Cho, Byung-Jun Lee
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| learn band-based retrieval of a contiguous quantile interval instead of top-k similarity | LDAR · LaRA overall, 32K, open-source avg | score: 62.12 (RAG) -> 70.00 | Improved |
| retrieve a learned subset of passages instead of the full long context | open-source LLMs · LaRA overall, 128K | score: 43.65 -> 61.55 | Improved |
| retrieve a learned subset of passages instead of the full long context | closed-source LLMs · LaRA overall, 32K | score: 74.00 -> 79.42 | Improved |
| LDAR retrieves more passages than RAG on the Hallucination task | LDAR · LaRA Hallucination, 32K, open-source avg | score: 81.5 (RAG) -> 76.5 | Hurt |
| replace independent Bernoulli passage selection with band-based selection (ablation) | LDAR · LaRA (training curves) | performance and passage ratio | Improved |
| add a token-count cost penalty to the training objective (ablation) | LDAR · LaRA Reasoning, Qwen-2.5-7B | accuracy | Hurt |
| apply LDAR trained on LaRA to HELMET zero-shot, versus long-context | Qwen-2.5-7B · HELMET HotpotQA | score: 30.0 -> 64.0 | Improved |
| apply LDAR zero-shot versus RAG on Mistral-Nemo-12B HotpotQA | Mistral-Nemo-12B · HELMET HotpotQA | score: 63.0 (RAG) vs 61.0 | No clear effect |
| train LDAR on bge embeddings and test with gte embeddings (ablation) | LDAR · LaRA average | score: 60.1 -> 57.7 | Hurt |
| use LDAR with transferred embedder versus RAG (ablation) | LDAR · LaRA average, gte -> bge | score: 53.9 -> 55.9 | Improved |
| add Self-Route or Adaptive-k heuristics to balance RAG and long context | baselines · LaRA overall, 32K open-source | score: 58.62 (LC) -> 59.45 (Self-Route), 51.70 (Adaptive-k) | No clear effect |
Closest papers by meaning
- Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG (ICLR 2025 · similarity 0.92)
- Inference Scaling for Long-Context Retrieval Augmented Generation (ICLR 2025 · similarity 0.91)
- Retrieval meets Long Context Large Language Models (ICLR 2024 · similarity 0.91)
- Adaptive Distraction: Probing LLM Contextual Robustness with Automated Tree Search (NeurIPS 2025 · similarity 0.90)
- Accelerating Inference of Retrieval-Augmented Generation via Sparse Context Selection (ICLR 2025 · similarity 0.90)
- RAPID: Long-Context Inference with Retrieval-Augmented Speculative Decoding (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
