Research map / Large language models / Retrieval and knowledge

DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation

NeurIPS 2025 · Large language models › Retrieval and knowledge · approach: RAG robustness and evaluation · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DynamicRAG, where an LLM reranker dynamically selects both the order and number of retrieved documents, trained by behavior cloning then DPO using LLM-output-quality rewards.

Problem

In RAG, choosing the number of retrieved documents k for the generator is hard: too few miss information, too many add noise; static rerankers use a fixed k.

Approach

Model the reranker as an RL agent that outputs a reordered, variable-size document subset, with rewards computed from the downstream generator's response quality (EM, BERTScore, ROUGE, length penalty, LLM-based score), optimized via DPO over sampled trajectories after an SFT cold start.

Key results

• DynamicRAG (LLaMA3-8B) reaches 48.4 EM on NQ and 36.7 on HotpotQA and 56.3 on ASQA, with 34.2 on 2WikiMultihopQA, outperforming LLaMA3-8B-based baselines except RankRAG on NQ and TriviaQA.
• On reranking recall, DynamicRAG (LLaMA3-8B, ~20k training samples) gets average recall 73.7 vs RankRAG 72.5 across NQ and HotpotQA.
• Adding the dynamic reranker to GPT-4o raises NQ EM from 40.0 to 42.3.

Measured effects reported in the paper

Surprising findings

• Removing the EM reward drops NQ EM from 48.4 to 39.6 and HotpotQA from 36.7 to 22.3, the largest ablation effect.
• The paper reports DynamicRAG does not outperform RankRAG on three benchmarks in the main table, despite using less training data.

Limitations

• Only list-wise reranking explored; point-wise and pair-wise not studied.
• Only DPO used as RL algorithm; PPO/GRPO not tested.
• Step-level or observation-based optimization left for future work; max of 15 documents restricts the dynamic range of k.

Experimental setup

Datasets: Alpaca KILT NQ FEVER HotpotQA ELI5 TriviaQA ASQA OpenBookQA MS MARCO

Benchmarks: NQ TriviaQA HotpotQA 2WikiMultihopQA ASQA FEVER ELI5

Metrics: exact match accuracy ROUGE-L Recall@5 Recall@10 Recall@20

Baselines: GPT-3.5-Turbo GPT-4 GPT-4o IRCoT ReAct FLARE RA-DIT Reward-RAG Self-RAG DRAGIN Smart-RAG RankRAG

Builds on: DPO RankRAG Self-RAG monoT5

Closest papers by meaning

• Adapting to Non-Stationary Environments: Multi-Armed Bandit Enhanced Retrieval-Augmented Generation on Knowledge Graphs (AAAI 2025 · similarity 0.90)
• Long-Context LLMs Meet RAG: Overcoming Challenges for Long Inputs in RAG (ICLR 2025 · similarity 0.90)
• Attention in Large Language Models Yields Efficient Zero-Shot Re-Rankers (ICLR 2025 · similarity 0.90)
• RankRAG: Unifying Context Ranking with Retrieval-Augmented Generation in LLMs (NeurIPS 2024 · similarity 0.90)
• C-3PO: Compact Plug-and-Play Proxy Optimization to Achieve Human-like Retrieval-Augmented Generation (ICML 2025 · similarity 0.89)
• Improving Retrieval-Augmented Generation through Multi-Agent Reinforcement Learning (NeurIPS 2025 · similarity 0.89)