Research map / Large language models / Retrieval and knowledge
FrugalRAG: Less is More in RL Finetuning for Multi-hop Question Answering
Abhinav Java, Srivathsan Koundinyan, Nagarajan Natarajan, Amit Sharma
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RL stage 2 (learn to stop) after exploration SFT (ablation) | FrugalRAG-Explore · HotPotQA (ColBERTv2) | searches per question: 5.99 -> 2.05 | Improved |
| add RL stage 2 after exploration SFT (ablation) | FrugalRAG-Explore · MuSiQue (ColBERTv2) | MBE accuracy: 30.10 -> 33.72 | Improved |
| add RL stage 2 after exploration SFT (ablation) | FrugalRAG-Explore · HotPotQA (ColBERTv2) | recall: 86.40 -> 82.80 | Hurt |
| replace SFT-with-finish baseline by RL stage 2 (ablation) | SFT (with finish) · 2Wiki (E5-base-v2) | efficiency tradeoff: 13.85 -> 18.02 | Improved |
| replace SFT-with-finish by RL stage 2 (ablation) | SFT (with finish) · HotPotQA (E5-base-v2) | efficiency tradeoff: 22.31 -> 22.30 | No clear effect |
| add stage-1 SFT before RL (ablation) | FrugalRAG · HotPotQA | recall: 66.5 -> 70.4 | Improved |
| add late-stop penalty to reward (ablation) | FrugalRAG · HotPotQA | avg searches: 3.144 -> 2.89 | Improved |
| jointly optimize query generation and stopping with recall reward (ablation) | FrugalRAG · HotPotQA | recall: 70.4 -> 68.3 | Hurt |
| RL directly on base model with recall reward (ablation) | FrugalRAG · HotPotQA | recall: 70.4 -> 66.8 | Hurt |
| swap answer generator Qwen2.5-7B for Qwen2.5-32B | FrugalRAG · HotPotQA (E5-base-v2) | MBE accuracy: 58.5 -> 61.4 | Improved |
| swap Qwen2.5-7B-Instruct base for Llama3.1-8B-Instruct (ablation) | FrugalRAG · HotPotQA | recall: 70.40 -> 70.00 | No clear effect |
| scale training examples from 100 to 500 (ablation) | FrugalRAG · HotPotQA | recall: 67.9 -> 70.8 | Improved |
Closest papers by meaning
- LoongRL: Reinforcement Learning for Advanced Reasoning over Long Contexts (ICLR 2026 · similarity 0.89)
- Thinker: Learning to Think Fast and Slow (NeurIPS 2025 · similarity 0.89)
- AceSearcher: Bootstrapping Reasoning and Search for LLMs via Reinforced Self-Play (NeurIPS 2025 · similarity 0.88)
- Learning from Synthetic Data Improves Multi-hop Reasoning (ICLR 2026 · similarity 0.88)
- Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning (ICLR 2025 · similarity 0.88)
- Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimization (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
