Research map / Multi-agent systems and games / LLM agents
TodoEvolve: Learning to Architect Agent Planning Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TodoEvolve, a meta-planner (Qwen3-14B trained with SFT then IGPO) that synthesizes task-specific planning systems within the PlanFactory codebase, evaluated on GAIA, WebWalkerQA, xBench-DS and TaskCraft.
Problem
Existing LLM agent planning systems use fixed, hand-crafted planning structures that do not adapt to the structural diversity of open-ended, long-horizon tasks.
Approach
Train a meta-planner to generate a query-specific planning system (topology, initialization, adaptation, navigation) as executable code, using a bootstrap-and-filter data pipeline with execution-verified SFT and a DPO-style preference step (IGPO) whose tie-breaker is a cost-based "cognitive impedance" metric.
Key results
- Smolagents with GPT-5-Mini plus TodoEvolve reaches 72.12% GAIA average, a reported 16.37-point gain over vanilla Smolagents.
- With DeepSeek V3.2, TodoEvolve+Smolagents reaches 70.91% GAIA average and 53.85% on GAIA Level 3.
- On the efficiency comparison with Kimi K2, TodoEvolve reaches 70.00% on WebWalker-QA and 42.00% on DeepSearch-QA, above the best baselines.
Surprising findings
- The base Qwen3-14B fails to synthesize executable plans without SFT, and zero-shot IGPO reduces API cost relative to SFT-only while improving accuracy.
Limitations
- Evaluation subsets are small (e.g., 50 DeepSearchQA queries, 170 WebWalkerQA queries, 100 xBench-DS tasks), so variance may be high.
- Cognitive impedance is a hand-designed compound cost with unspecified weights and the claimed gains partly rely on LLM-as-judge scoring.
Experimental setup
Datasets: TaskCraft MoNaCo WebWalkerQA DeepSearchQA GAIA xBench-DS
Benchmarks: GAIA xBench-DS TaskCraft WebWalkerQA DeepSearchQA
Metrics: pass@1 accuracy average cost ($) average time (s) average steps
Baselines: OWL OAgents AgentOrchestra Flash-Searcher JoyAgent FlowSearch Co-Sight Smolagents Agent KB Cognitive Kernel-Pro
Builds on: DPO Plan-and-Act Flash-Searcher OWL Smolagents
Closest papers by meaning
- Agent-Oriented Planning in Multi-Agent Systems (ICLR 2025 · similarity 0.87)
- Automated Design of Agentic Systems (ICLR 2025 · similarity 0.87)
- SPIRAL: Symbolic LLM Planning via Grounded and Reflective Search (AAAI 2026 · similarity 0.86)
- Plan-and-Act: Improving Planning of Agents for Long-Horizon Tasks (ICML 2025 · similarity 0.86)
- MemEvolve: Meta-Evolution of Agent Memory Systems (ICML 2026 · similarity 0.86)
- ToolChain*: Efficient Action Space Navigation in Large Language Models with A* Search (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.