Research map / Multi-agent systems and games / LLM agents
Internet of Agents: Weaving a Web of Heterogeneous Agents for Collaborative Intelligence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Internet of Agents (IoA), an instant-messaging-style framework that lets heterogeneous third-party agents register, discover each other, form nested teams, and collaborate in group chats under LLM-driven finite-state conversation control.
Problem
Existing multi-agent frameworks isolate agents within their own ecosystems, run on a single device, and rely on hard-coded communication pipelines, limiting integration of heterogeneous third-party agents and adaptability to dynamic tasks.
Approach
An Internet-inspired client-server architecture with an agent integration protocol (run(task_desc) -> str), semantic agent discovery, autonomous nested team formation, and an LLM-controlled finite state machine over discussion, sync/async task assignment, pause-and-trigger, and conclusion states.
Key results
- On the GAIA validation set, IoA with four basic ReAct agents reaches 40.00% overall, versus 39.39% for AutoGen and 14.60% for GPT-4 + Plugins.
- On the open-ended instruction benchmark, IoA wins 76.5% against AutoGPT and 63.4% against Open Interpreter under GPT-4 judging with order alternation.
- On RAG, IoA with 3 GPT-3.5 agents (homogeneous) reaches 0.671 overall accuracy, above GPT-4 at 0.611.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| orchestrate four ReAct agents with different tools using IoA | IoA · GAIA validation, Overall | accuracy: 39.39 -> 40.00 | Helped |
| orchestrate AutoGPT and Open Interpreter through IoA | AutoGPT and Open Interpreter · Open-ended instructions | win rate: 76.5% against AutoGPT; 63.4% against Open Interpreter | Helped |
| use IoA collaboration for embodied agents | RoCoBench · RoCoBench Cabinet | success rate: 0.75 -> 1.00 | Helped |
| use IoA collaboration for embodied agents | RoCoBench · RoCoBench Rope | success rate: 0.70 -> 0.70 | No clear effect |
| use 3 homogeneous agents instead of 2 homogeneous agents (ablation) | IoA RAG · RAG Overall | accuracy: 0.652 -> 0.671 | Helped |
| use 2 homogeneous agents instead of 3 homogeneous agents (ablation) | IoA RAG · HotpotQA | accuracy: 0.575 -> 0.586 | Helped |
| use 2 homogeneous agents instead of 3 homogeneous agents (ablation) | IoA RAG · 2WikiMultiHopQA | accuracy: 0.519 -> 0.530 | Helped |
| use 2 agents with different evidence pools instead of 2 agents with access to both pools (ablation) | IoA RAG · NQ | accuracy: 0.671 -> 0.708 | Helped |
| use 2 agents with different evidence pools instead of 2 agents with access to both pools (ablation) | IoA RAG · HotpotQA | accuracy: 0.586 -> 0.478 | Hurt |
| remove repeated or rephrased chat messages (ablation) | IoA · Open-ended instructions | cost per task: $0.99 -> $0.74 | Helped |
Surprising findings
- Deduplicating repeated LLM communication cut communication cost by nearly 50%, indicating agents are not efficient communicators despite being good chatbots.
- A 2-agent homogeneous RAG setup outperformed the 3-agent setup on HotpotQA and 2WikiMultiHopQA, suggesting task-dependent optimal team size.
Limitations
- Communication cost is high: about $0.99 per task overall, with LLMs often failing to switch to pause-and-trigger and repeating content.
- The Security Module is not extensively implemented, and third-party agent verification is not addressed.
- GAIA evaluated only on the validation set due to budget; open-ended benchmark is small (153 tasks) and judged by an LLM.
Experimental setup
Datasets: GAIA RoCoBench TriviaQA Natural Questions HotpotQA 2WikiMultiHopQA XAgent tasks
Benchmarks: GAIA validation Open-ended instruction benchmark (153 tasks) RoCoBench TriviaQA Natural Questions HotpotQA 2WikiMultiHopQA
Metrics: accuracy win rate success rate number of steps Top@1 Top@10 Mean Rank MRR cost per task
Baselines: GPT-4 GPT-4-Turbo AutoGPT GPT-4 + Plugins FRIDAY AutoGen Central Plan Roco Dialog Zero-Shot CoT Self Consistency Reflexion Multi-Agent Debate
Builds on: ReAct AutoGen Apollo's Oracle Speech Act Theory Self-Instruct
Closest papers by meaning
- AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors (ICLR 2024 · similarity 0.88)
- Autonomous Agents for Collaborative Task under Information Asymmetry (NeurIPS 2024 · similarity 0.87)
- Building Cooperative Embodied Agents Modularly with Large Language Models (ICLR 2024 · similarity 0.87)
- WebArena: A Realistic Web Environment for Building Autonomous Agents (ICLR 2024 · similarity 0.87)
- AgentSwift: Efficient LLM Agent Design via Value-Guided Hierarchical Search (AAAI 2026 · similarity 0.87)
- AgentSquare: Automatic LLM Agent Search in Modular Design Space (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.