Research map / Multi-agent systems and games / LLM agents
Agent-Oriented Planning in Multi-Agent Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AOP, a multi-agent planning framework that decomposes and allocates user queries with a meta-agent, then refines plans using a learned reward model, a detector for completeness/non-redundancy, representative works, and a feedback loop.
Problem
Meta-agents in LLM multi-agent systems often produce sub-optimal task decomposition and allocation, failing to ensure sub-tasks are solvable by assigned agents, complete, and non-redundant.
Approach
Identifies three design principles (solvability, completeness, non-redundancy) and operationalizes them via a fast decompose-and-allocate step followed by a lightweight MLP reward model over sentence embeddings that predicts agent suitability without executing agents, plus an LLM detector and representative-works similarity for re-describing or planning in detail.
Key results
- AOP reaches 43.7% accuracy on Husky-QA, versus 39.6% for HUSKY and 37.6% for REACT, and 33.3% for GPT-4o.
- On DROP and IIRC, AOP scores 34.0% and 41.5%, improving over the best baseline by at least 5 points.
- Adding manual scoring and full parameter tuning of the reward model raises Husky-QA accuracy to 46.9% and 47.9%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AOP planning instead of GPT-4o alone | AOP · Husky-QA test | accuracy (%): 33.3 -> 43.7 | Helped |
| use AOP planning instead of HUSKY | AOP · Husky-QA test | accuracy (%): 39.6 -> 43.7 | Helped |
| use AOP planning instead of REACT | AOP · Husky-QA test | accuracy (%): 37.6 -> 43.7 | Helped |
| use AOP instead of the strongest reported baseline | AOP · DROP | accuracy (%): 29.0 -> 34.0 | Helped |
| use AOP instead of the strongest reported baseline | AOP · IIRC | accuracy (%): 36.5 -> 41.5 | Helped |
| use AOP planning with multiple math agents instead of GPT-4o | AOP · MATH | accuracy (%): 62 -> 72 | Helped |
| add the Plan Detector (ablation) | AOP · Husky-QA test | accuracy (%): 36.6 -> 43.7 | Helped |
| add the Reward Model (ablation) | AOP · Husky-QA test | accuracy (%): 38.7 -> 43.7 | Helped |
| add Representative Works (ablation) | AOP · Husky-QA test | accuracy (%): 41.1 -> 43.7 | Helped |
| use human manual scoring to train the reward model | AOP · Husky-QA test | accuracy (%): 43.7 -> 46.9 | Helped |
| update the reward model embedding layers with full parameter tuning | AOP · Husky-QA test | accuracy (%): 43.7 -> 47.9 | Helped |
| combine natural language descriptions with representative works instead of using either alone | AOP · Husky-QA | accuracy (%): 31.8 -> 43.7; 16.8 -> 43.7 | Helped |
Surprising findings
- Meta-Agent with GPT-4o alone (30.0%) underperforms plain GPT-4o (33.3%) on Husky-QA.
- Representative works alone reach only 16.8% accuracy, versus 31.8% for natural-language descriptions alone.
Limitations
- Relies on GPT-4o as meta-agent, scorer, and evaluator, so results depend on a closed model.
- Evaluated mainly on Husky-QA-style numerical reasoning with a small set of four agents.
- Higher token and time costs than single-agent baselines.
Experimental setup
Datasets: Husky-QA DROP IIRC MATH
Benchmarks: Husky-QA DROP IIRC MATH
Metrics: accuracy prompt tokens completion tokens execution time
Baselines: GPT-4o CoT Zero-Shot CoT Meta-Agent Meta-Agent: Traversal REACT HUSKY
Builds on: ReAct HUSKY HuggingGPT Chain-of-Thought prompting
Closest papers by meaning
- TodoEvolve: Learning to Architect Agent Planning Systems (ICML 2026 · similarity 0.87)
- Multi-Agent Design: Optimizing Agents with Better Prompts and Topologies (ICLR 2026 · similarity 0.87)
- Automated Design of Agentic Systems (ICLR 2025 · similarity 0.87)
- MoRSE: Task-Oriented Multi-Agent System with Mixture of Role-Subtask Experts (NeurIPS 2026 · similarity 0.86)
- Adaptive Collaboration with Humans: Metacognitive Policy Optimization for Multi-Agent LLMs with Continual Learning (ICLR 2026 · similarity 0.85)
- Automated Composition of Agents: A Knapsack Approach for Agentic Component Selection (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.