Research map / Large language models / Retrieval and knowledge
Scent of Knowledge: Optimizing Search-Enhanced Reasoning with Information Foraging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes InForage, an RL framework that trains search-augmented reasoning LLMs with outcome, information-gain, and efficiency rewards inspired by Information Foraging Theory, plus a human-guided web-browsing trajectory dataset.
Problem
Static pre-inference retrieval in RAG is inadequate for complex multi-step, ambiguous, or evolving information needs; outcome-only rewards give sparse signals for intermediate retrieval quality.
Approach
Model search-enhanced reasoning as information foraging: rewards combine final answer correctness, maximum coverage of golden evidence patches during the trajectory (information gain), and an exponential efficiency penalty on the number of steps, optimized with PPO after SFT on annotated trajectories.
Key results
- InForage (Qwen2.5-3B) averages 41.0 EM across eight datasets, versus 35.6 for the best baseline Search-R1-GRPO.
- On the self-constructed Self dataset, InForage reaches 44.1 EM versus 36.7 for Search-o1.
- Ablations show removing the information gain reward causes consistent drops; PPO generally beats GRPO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add supervised fine-tuning before reinforcement learning (ablation) | InForage · benchmarks | performance | Helped |
| include the self-constructed dataset in training (ablation) | InForage · benchmarks | performance | Helped |
| add the information gain reward (ablation) | InForage · benchmarks | performance | Helped |
| add the efficiency penalty (ablation) | InForage · benchmarks except Self | performance | Helped |
| add the efficiency penalty (ablation) | InForage · Self | performance | No clear effect |
| use PPO instead of GRPO (ablation) | InForage · benchmarks | performance | Mixed |
| replace the Qwen2.5-3B foundation model with Qwen2.5-7B (ablation) | InForage · most benchmarks | performance | Helped |
Surprising findings
- Removing the efficiency penalty improved results on the self-constructed dataset, suggesting complex web tasks may need longer reasoning chains.
Limitations
- Experiments limited to Qwen2.5 3B and 7B models; other model families not explored.
- Only short-form QA with verifiable answers is evaluated; long-form tasks not explored.
- Some concurrent search-reasoning baselines were not included in comparisons.
Experimental setup
Datasets: Natural Questions HotpotQA Self (self-constructed web QA) FlashRAG Wikipedia dump Google Search results cache
Benchmarks: Natural Questions TriviaQA PopQA HotpotQA 2WikiMultihopQA MuSiQue Bamboogle Self
Metrics: exact match
Baselines: Vanilla SFT Reasoning RL Vanilla RAG IRCoT RQRAG Self-RAG Search-o1 Search-R1-PPO Search-R1-GRPO
Builds on: Information Foraging Theory PPO Search-R1 Search-o1 IRCoT
Closest papers by meaning
- Adapting to Non-Stationary Environments: Multi-Armed Bandit Enhanced Retrieval-Augmented Generation on Knowledge Graphs (AAAI 2025 · similarity 0.89)
- Iterative Self-Incentivization Empowers Large Language Models as Agentic Searchers (NeurIPS 2025 · similarity 0.89)
- Empowering Efficiency and Efficacy in WebAgent via Enabling Info-Rich Seeking (ICLR 2026 · similarity 0.89)
- DynamicRAG: Leveraging Outputs of Large Language Model as Feedback for Dynamic Reranking in Retrieval-Augmented Generation (NeurIPS 2025 · similarity 0.88)
- Search and Refine During Think: Facilitating Knowledge Refinement for Improved Retrieval-Augmented Reasoning (NeurIPS 2025 · similarity 0.88)
- DeepDiver: Adaptive Web-Search Intensity Scaling via Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.