Research map / Multi-agent systems and games / LLM agents
IterResearch: Rethinking Long-Horizon Agents via Markovian State Reconstruction
Guoxin Chen, Zile Qiao, Xuanzhong Chen, Donglei Yu, Haotian Xu, Xin Zhao, Ruihua Song, Wenbiao Yin, Huifeng Yin, Liwen Zhang, Kuan Li, Minpeng Liao and 4 more
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IterResearch, a deep-research agent that replaces a single ever-expanding context with iterative Markovian state reconstruction to handle long-horizon tasks.
Problem
Deep-research agents that accumulate all information in one expanding context window suffer from context suffocation and noise contamination on long-horizon tasks.
Approach
Replaces the mono-contextual paradigm with iterative rounds in which the agent reconstructs a compact Markovian state from its previous state rather than appending everything to one growing context.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use iterative workspace reconstruction instead of mono-contextual accumulation (same data, mono gets 64K vs 40K context) (ablation) | IterResearch agent · six deep-research benchmarks (avg) | accuracy: 36.5 -> 49.1 average (BC 25.4 -> 37.3) | Improved |
| add efficiency-aware geometric discounted reward (EAPO) over GSPO (ablation) | IterResearch · six benchmarks avg | accuracy and avg turns: 48.3 -> 49.1 accuracy; 19.13 -> 18.04 turns | Improved |
| add RL (GSPO/EAPO) after SFT (ablation) | IterResearch · six benchmarks avg | accuracy: 45.5 -> 49.1 | Improved |
| mix iterative-paradigm trajectories into mono-agent training data (ablation) | Mono-Agent · six benchmarks avg | accuracy: 36.5 -> 41.9 | Improved |
| raise maximum interaction turns from 2 to 2048 (ablation) | IterResearch · BrowseComp (200 subset) | accuracy: 3.5 -> 42.5 (text says 5.5% -> 50.1%, disagreeing with figure) | Improved |
| apply IterResearch paradigm as prompt instead of ReAct | o3 · BrowseComp | accuracy: 49.7 -> 62.4 | Improved |
| apply IterResearch paradigm as prompt instead of ReAct | DeepSeek-V3.1 · BrowseComp | accuracy: 30.0 -> 49.2 | Improved |
| EAPO vs GSPO on BrowseComp (ablation) | IterResearch · BrowseComp | accuracy: 38.3 -> 37.3 | No clear effect |
Closest papers by meaning
- ReLaX: Reasoning with Latent Exploration for Large Reasoning Models (CVPR 2026 · similarity 0.89)
- Search More, Think Less: Rethinking Long-Horizon Agentic Search for Efficiency and Generalization (NeurIPS 2026 · similarity 0.89)
- DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search (ICLR 2026 · similarity 0.89)
- SD-Search: Hindsight Self-Distillation for Search-Augmented Reasoning (NeurIPS 2026 · similarity 0.88)
- Meta-Cognitive Memory Policy Optimization for Long-Horizon LLM Agents (NeurIPS 2026 · similarity 0.87)
- MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
