AI Paper Atlas

Research map / Multi-agent systems and games / LLM agents

Unlocking Long-Horizon Agentic Search with Large-Scale End-to-End RL

Jiaxuan Gao, Wei Fu, Minyang Xie, Shusheng Xu, Chuyi He, Zhiyu Mei, Banghua Zhu, Yi Wu

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a large-scale end-to-end RL recipe for training long-horizon LLM search agents without reliance on commercial-LLM trajectories or distillation.

Problem

Open-source search agents depend heavily on advanced commercial LLMs, either via SFT on their trajectories or by calling them as tools, limiting autonomy and long-horizon search capability.

Approach

Trains search agents end-to-end with large-scale reinforcement learning instead of imitating trajectories from stronger commercial models, aiming for long-horizon agentic search built from the agent's own rollouts.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
end-to-end RL with 128-turn limitQwQ-32B search agent · GAIALLM-as-Judge Pass@1: text says +15.0; Table 1 gives 58.7Improved
raise training turn limit from 10 to 128 (ablation)ASearcher · GAIALLM-as-Judge Pass@1: 49.2 -> 58.7Improved
raise training turn limit from 10 to 128 (ablation)ASearcher · xBench-DeepSearchLLM-as-Judge Pass@1: 39.3 -> 51.1Improved
add stage-2 curriculum on questions needing >=5 tool calls (ablation)ASearcher · GAIALLM-as-Judge Pass@1: 51.6 -> 58.7Improved
add stage-2 curriculum on questions needing >=5 tool calls (ablation)ASearcher · xBench-DeepSearchLLM-as-Judge Pass@1: 43.0 -> 51.1Improved
train on synthesized QA data instead of AFM data (ablation)ASearcher · GAIALLM-as-Judge Pass@1: 50.9 -> 58.7Improved
train on synthesized QA data instead of AFM data (ablation)ASearcher · xBench-DeepSearchLLM-as-Judge Pass@1: 39.9 -> 51.1Improved
use DeepSeek-V3 as external webpage summarizerASearcher · GAIALLM-as-Judge Pass@1: 58.7 -> 60.3Improved
add test-time search aggregating K=16 rolloutsASearcher · GAIALLM-as-Judge Pass@1: 60.3 -> 71.8Improved
add test-time search aggregating K=16 rolloutsASearcher · xBench-DeepSearchLLM-as-Judge Pass@1: 56.4 -> 75.0Improved
apply same RL recipe to a small non-reasoning modelQwen2.5-7B-Instruct · GAIALLM-as-Judge Pass@1: 18.2 -> 27.7Improved
apply same RL recipe to a small non-reasoning modelQwen2.5-7B-Instruct · xBench-DeepSearchLLM-as-Judge Pass@1: 23.0 -> 27.8Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy