Research map / Large language models / Retrieval and knowledge
Iterative Self-Incentivization Empowers Large Language Models as Agentic Searchers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ExSearch, an agentic search framework where an LLM interleaves thinking, search, and recording actions and is trained via a Generalized EM loop with importance-weighted self-generated trajectories.
Problem
Enabling LLMs to seek accurate knowledge for complex multi-hop queries despite irrelevant retrieved content, with end-to-end alignment of retrieval stages.
Approach
Treat search trajectories (sub-queries, retrieved docs, extracted evidence) as latent variables and train the LLM with a generalized EM loop: sample trajectories from the current model, weight each by p(y|x,z), and maximize the re-weighted ELBO, yielding a self-incentivized loop.
Key results
- Ours-Qwen-2.5-7B averages 51.45 F1 / 43.02 EM / 46.20 Acc across four benchmarks, versus Search-R1 at 48.74 F1 / 39.89 EM / 44.02 Acc.
- On HotpotQA, Ours-Qwen-2.5-7B reaches 62.59 F1, compared with 53.31 for Search-o1 (QwQ-32B) and 55.40 for RankRAG (70B).
- Removing the importance weight w(z) drops HotpotQA F1 from 60.72 to 54.45 for the Llama-3.1-8B variant.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ExSearch instead of Search-R1 | Qwen2.5-7B · NQ | F1: 54.26 -> 56.37 | Helped |
| Use ExSearch instead of IRCoT | HotpotQA · HotpotQA | F1: 58.40 -> 62.59 | Helped |
| Use ExSearch instead of IRCoT | MuSiQue · MuSiQue | F1: 30.50 -> 30.83 | Helped |
| Use ExSearch instead of Search-R1 | Qwen2.5-7B · 2WikiQA | F1: 52.62 -> 57.14 | Helped |
| Add thinking (iterative query refinement) (ablation) | ExSearch-Qwen2.5-7B and ExSearch-Llama3.1-8B · HotpotQA | F1: 54.23 -> 62.59; 52.31 -> 60.72 | Helped |
| Add external search (ablation) | ExSearch-Qwen2.5-7B and ExSearch-Llama3.1-8B · HotpotQA | F1: 43.34 -> 62.59; 48.25 -> 60.72 | Helped |
| Add fine-grained evidence recording (ablation) | ExSearch-Qwen2.5-7B and ExSearch-Llama3.1-8B · HotpotQA | F1: 57.34 -> 62.59; 55.23 -> 60.72 | Helped |
| Add trajectory importance weighting w(z) (ablation) | ExSearch-Qwen2.5-7B and ExSearch-Llama3.1-8B · HotpotQA | F1: 57.23 -> 62.59; 54.45 -> 60.72 | Helped |
Surprising findings
- Qwen2.5-7B without any cold-start warm-up data already outperforms strong baselines, per the authors.
- Plain query decomposition methods like Search-R1 have comparable recall to the proposed method on some datasets, and re-ranking methods give limited multi-hop gains.
Limitations
- Text-only inputs and outputs; multimodal extension not explored.
- Retrieves at every step regardless of necessity; no adaptive stopping or retrieval decision.
- Relies on answer-centric metrics (EM/F1) and gold answers for supervision.
Experimental setup
Datasets: Natural Questions HotpotQA MuSiQue 2WikiMultihopQA Wikipedia dump (Dec 20, 2018)
Benchmarks: Natural Questions HotpotQA MuSiQue 2WikiMultihopQA
Metrics: F1 Exact Match Accuracy Recall@3 Recall@5
Baselines: Search-R1 Search-o1 Self-RAG RankRAG ChatQA IRCoT GenGround DSPy InstructRAG GPT-4o
Builds on: Search-R1 Self-RAG Generalized EM importance sampling
Closest papers by meaning
- Demystifying and Enhancing the Efficiency of Large Language Model Based Search Agents (ICLR 2026 · similarity 0.90)
- Empowering Efficiency and Efficacy in WebAgent via Enabling Info-Rich Seeking (ICLR 2026 · similarity 0.89)
- MC-Search: Evaluating and Enhancing Multimodal Agentic Search with Structured Long Reasoning Chains (ICLR 2026 · similarity 0.89)
- WideSearch: Benchmarking Agentic Broad Info-Seeking (ICLR 2026 · similarity 0.89)
- Scent of Knowledge: Optimizing Search-Enhanced Reasoning with Information Foraging (NeurIPS 2025 · similarity 0.89)
- Language Models can Self-Improve at State-Value Estimation for Better Search (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.