Research map / Multi-agent systems and games / LLM agents
AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AutoManual, where a Planner (code-writing LLM agent) and a Builder alternate online to build a structured rule system from interaction, which a Formulator compiles into a Markdown manual that guides smaller LLMs on ALFWorld, MiniWoB++ and WebArena.
Problem
LLM agents in unfamiliar environments need elaborate expert prompts; existing skill/reflection libraries and offline rule extraction (ExpeL, AutoGuide) cause path dependence and distribution shift.
Approach
Online alternating optimization of a structured, typed rule system (type, content, example, logs) by a Builder agent that reads code-based Planner trajectories, with case-conditioned prompting to reduce hallucinated rules, followed by manual formulation; rules are updated in text rather than by gradient updates.
Key results
- AutoManual reaches 97.4% success on ALFWorld with GPT-4-turbo and 86.2% with GPT-3.5-turbo testing, versus 85.9% (Reflexion) and 59.8% (Reflexion) baselines respectively.
- On MiniWoB++ 9 feedback tasks, AutoManual achieves 94.5% (GPT-4-turbo) and 82.2% (GPT-3.5-turbo); on all 53 tasks 98.3% and 92.7%.
- On WebArena Reddit, AutoManual reaches 65.1% success versus 55.0% for SteP and 43.7% for AutoGuide, using one demonstration.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AutoManual rather than Reflexion | GPT-3.5-turbo Planner · ALFWorld unseen validation | success rate: 59.8 -> 86.2 | Helped |
| Use AutoManual rather than Reflexion | GPT-4-turbo Planner · ALFWorld unseen validation | success rate: 85.9 -> 97.4 | Helped |
| Use AutoManual rather than AdaPlanner | GPT-3.5-turbo Planner · MiniWoB++ 9 task types with feedback | success rate: 71.6 -> 82.2 | Helped |
| Use AutoManual rather than Planner+Lib. | GPT-4-turbo Planner · MiniWoB++ 9 task types with feedback | success rate: 80.2 -> 94.5 | Helped |
| Use online rather than offline rule management (ablation) | AutoManual · ALFWorld | success rate: 90.7 -> 97.4 | Helped |
| Add Skill&Reflection Libraries (ablation) | AutoManual with online management, Case Prompt, and Formulation · ALFWorld | success rate: 89.5 -> 97.4 | Helped |
| Add Case Prompt (ablation) | AutoManual with online management, Skill&Reflect Lib., and Formulation · ALFWorld | success rate: 93.8 -> 97.4 | Helped |
| Add Formulation (ablation) | AutoManual with online management, Skill&Reflect Lib., and Case Prompt · ALFWorld | success rate: 96.5 -> 97.4 | Helped |
| Add rule Type attribute and type-specific Builder instructions (ablation) | AutoManual · ALFWorld | success rate: 74.6 -> 86.2; 91.5 -> 97.4 | Helped |
| Add rule Example attribute (ablation) | AutoManual · ALFWorld | success rate: 76.8 -> 86.2; 92.7 -> 97.4 | Helped |
| Add rule Validation Logs (ablation) | AutoManual · ALFWorld | success rate: 85.4 -> 86.2; 97.0 -> 97.4 | No clear effect |
| Add Useful Helper Method support (ablation) | AutoManual · ALFWorld | success rate: 79.6 -> 86.2; 94.3 -> 97.4 | Helped |
Surprising findings
- Offline rule management (without online verification) only slightly improves planning (88.1% to 90.7%), suggesting online verification matters.
- Rule scores in ExpeL/AutoGuide were unreliable because the Builder gave overconfident scores to all rules.
Limitations
- Relies heavily on GPT-4-turbo capability to generate reliable rules.
- All rules are placed in context, which may not scale to larger environments; RAG suggested as remedy.
- Limited exploration; agent only attempts solutions based on current knowledge and Planner may ignore rules.
Experimental setup
Datasets: ALFWorld MiniWoB++ WebArena
Benchmarks: ALFWorld MiniWoB++ WebArena
Metrics: success rate average error steps
Baselines: ReAct Reflexion ExpeL RCI AdaPlanner Planner+Lib. AutoGuide SteP
Builds on: Reflexion Voyager AdaPlanner ExpeL AutoGuide
Closest papers by meaning
- Scaling Autonomous Agents via Automatic Reward Modeling And Planning (ICLR 2025 · similarity 0.89)
- Learn-by-interact: A Data-Centric Framework For Self-Adaptive Agents in Realistic Environments (ICLR 2025 · similarity 0.89)
- WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environment (NeurIPS 2024 · similarity 0.89)
- Test-Time Adaptation for LLM Agents via Environment Interaction (ICLR 2026 · similarity 0.88)
- SkillMaster: Toward Autonomous Skill Mastery in LLM Agents (NeurIPS 2026 · similarity 0.88)
- AgentSquare: Automatic LLM Agent Search in Modular Design Space (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.