Research map / Multi-agent systems and games / LLM agents
Hierarchical Procedural Meta-Reasoning for Generalizable Multimodal Agents
Yao Fu, Shengyi Qian, Pierluca D'Oro, Fanyi Xiao, Honglak Lee, Joseph Tighe, Manchen Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
The paper proposes hierarchical procedural meta-reasoning to improve generalization of multimodal agents on complex tasks such as mobile navigation; only the abstract was available.
Problem
Multimodal agents fine-tuned for tasks like mobile navigation generalize poorly across diverse applications, frequent system changes, and customized workflows, which the authors attribute to insufficient task-specific procedural knowledge.
Approach
Abstract indicates a hierarchical procedural meta-reasoning approach intended to supply agents with procedural knowledge for generalization; specific mechanism not stated in the available text.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| condition the actor on a static procedure generated once | PLM-8B actor · DigiData-Bench-Auto (all) | success rate (%): 41.47 -> 46.30 | Improved |
| update the procedure online with internal knowledge (ablation) | PAMAR-Static · DigiData-Bench-Auto (all) | success rate (%): 46.30 -> 52.10 | Improved |
| add prompt-based meta-reasoner with selective search (ablation) | PAMAR-Dynamic · DigiData-Bench-Auto (all) | success rate (%): 52.10 -> 54.20 | Improved |
| add selective external search via meta-reasoner on unseen apps (ablation) | PAMAR-Dynamic · DigiData-Bench-Auto (cross-app) | success rate (%): 40.51 -> 47.06 | Improved |
| always search instead of selective search (ablation) | PAMAR with PLM-8B · DigiData-Bench-Auto (all) | success rate (%): 54.20 -> 54.90 | No clear effect |
| add static procedure on updated app versions (ablation) | PLM-8B actor · DigiData-AppUpdate | success rate (%): 29.50 -> 30.35 | No clear effect |
| allow dynamic procedure revision on updated app versions (ablation) | PAMAR-Static · DigiData-AppUpdate | success rate (%): 30.35 -> 55.75 | Improved |
| add trained meta-reasoner with update notes (ablation) | PAMAR-Dynamic · DigiData-AppUpdate | success rate (%): 55.75 -> 63.93 | Improved |
| add static procedure on multi-app tasks (ablation) | PLM-8B actor · DigiData-MultiApp | success rate (%): 0.0 -> 0.0 | No clear effect |
| add dynamic procedure and meta-reasoner on multi-app tasks (ablation) | PLM-8B actor · DigiData-MultiApp | success rate (%): 0.0 -> 15.0 | Improved |
| delay the first predicted search trigger (ablation) | PAMAR (Train) · DigiData-Bench-Auto (all) | success rate (%): 53.47 -> 48.61 | Hurt |
| train the actor with augmented procedure paraphrases and abstraction instead of step-wise procedures only (ablation) | PLM-8B actor with dynamic procedures · DigiData-Bench-Auto (all) | success rate (%): 48.30 -> 52.10 | Improved |
Limitations
- Only the abstract was available; experimental details are unknown.
Closest papers by meaning
- Optimus-1: Hybrid Multimodal Memory Empowered Agents Excel in Long-Horizon Tasks (NeurIPS 2024 · similarity 0.87)
- Agent S: An Open Agentic Framework that Uses Computers Like a Human (ICLR 2025 · similarity 0.87)
- Mobile-Agent-RAG: Driving Smart Multi-Agent Coordination with Contextual Knowledge Empowerment for Long-Horizon Mobile Automation (AAAI 2026 · similarity 0.86)
- K²-Agent: Co-Evolving Know-What and Know-How for Hierarchical Mobile Device Control (ICLR 2026 · similarity 0.86)
- AgentVista: Evaluating Multimodal Agent in Ultra-Challenging Realistic Visual Scenarios (NeurIPS 2026 · similarity 0.86)
- Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaboration (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
