Research map / Multi-agent systems and games / LLM agents

Latent Collaboration in Multi-Agent Systems

ICML 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LatentMAS, a training-free multi-agent framework where LLM agents reason and communicate via last-layer hidden states and shared layer-wise KV-cache working memory instead of text.

Problem

Text-based mediation in LLM multi-agent systems is token-expensive and slow; the paper asks whether MAS can collaborate purely in continuous latent space.

Approach

Agents auto-regressively feed last-layer hidden states back as inputs (latent thoughts), aligned by a closed-form ridge-regression matrix W_a mapping output to input embedding space, and pass layer-wise KV caches to successor agents as lossless latent working memory, with no training.

Key results

• LatentMAS improves accuracy over single model by an average of 14.6% (sequential) and 13.3% (hierarchical), and over text-based MAS by 2.8% and 4.6%.
• LatentMAS reduces output token usage by 70.8% and 83.7% relative to sequential and hierarchical TextMAS.
• LatentMAS achieves about 4x and 4.3x faster end-to-end inference than sequential and hierarchical TextMAS on average.

Surprising findings

• On several tasks (e.g., ARC-Easy with Qwen3-8B, GSM8K with Qwen3-4B), LatentMAS is slightly below TextMAS in accuracy.

Limitations

• Accuracy gains over text-based MAS are small on several benchmarks and are sometimes negative.
• Latent thoughts are not human-readable; a debug mode is needed for interpretation.
• Performance plateaus or declines beyond roughly 40-80 latent steps.

Experimental setup

Datasets: GSM8K AIME24 AIME25 GPQA-Diamond MedQA ARC-Easy ARC-Challenge MBPP-Plus HumanEval-Plus

Benchmarks: GSM8K AIME24 AIME25 GPQA-Diamond MedQA ARC-Easy ARC-Challenge MBPP-Plus HumanEval-Plus

Metrics: accuracy output token usage end-to-end inference speed

Baselines: Single LLM Sequential TextMAS Hierarchical TextMAS

Builds on: chain-of-agents latent reasoning via continuous thoughts HuggingFace Transformers KV cache

Closest papers by meaning

• Agent Primitives: Reuseable Latent Building Blocks for Multi-Agent Systems (ICML 2026 · similarity 0.91)
• Thought Communication in Multiagent Collaboration (NeurIPS 2025 · similarity 0.90)
• Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs (ICML 2026 · similarity 0.89)
• G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems (NeurIPS 2025 · similarity 0.89)
• Chain of Agents: Large Language Models Collaborating on Long-Context Tasks (NeurIPS 2024 · similarity 0.89)
• AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors (ICLR 2024 · similarity 0.89)