Research map / Multi-agent systems and games / LLM agents

MOC: Multi-Order Communication in LLM-based Multi-Agent Systems

ICML 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Multi-Order Communication (MOC), which exposes a target LLM agent to raw upstream responses across multiple hop orders and consolidates them via embedding-based, distillation-driven merging to stay within a token budget.

Problem

LLM-based multi-agent systems typically pass only first-order neighbor outputs by direct concatenation, which restricts the evidence receptive field and dilutes multi-hop insights, while naively expanding context causes token blow-up.

Approach

Build a K-order message set by shortest hop distance over the DAG (via adjacency-matrix powers), order far-to-near under topological order, and consolidate redundant messages with a semantic-topological merging operator that uses embedding similarity and a small LM distiller under length constraints.

Key results

• On AQuA with Gemma-2-27B at edge density 0.3, MOC improves accuracy from 0.6969 to 0.7441 (+6.77% relative).
• On HumanEval with Qwen2.5-32B at edge density 0.3, MOC raises Pass@1 from 0.8293 to 0.8598 (+3.68%).
• With 20 agents, MOC reduces input tokens from 13.69e5 (w/o consolidation) to 12.49e5, below Vanilla MAS at 13.38e5.

Surprising findings

• Increasing the hop order to K=3 does not consistently help; at density 0.7 accuracy drops to 0.8354, below K=2.
• On G-Designer topologies, MOC increases total MAS input tokens by up to 40.1% on HumanEval because consolidation is rarely triggered.

Limitations

• Hop order K is fixed rather than adaptive.
• Multi-order exposure may propagate incorrect or malicious messages from compromised agents.
• Gains are small on several benchmarks (e.g., under 1%) and evaluated on small test subsets with few runs.

Experimental setup

Datasets: MMLU MMLU-Pro GSM8K SVAMP AQuA HumanEval

Benchmarks: MMLU MMLU-Pro GSM8K SVAMP AQuA HumanEval

Metrics: accuracy Pass@1 input tokens output tokens runtime

Baselines: Single LLM Vanilla MAS G-Designer MOC w/o Φ

Builds on: MixHop GNN message passing G-Designer LLM-Debate

Closest papers by meaning

• Graph-of-Agents: A Graph-based Framework for Multi-Agent LLM Collaboration (ICLR 2026 · similarity 0.90)
• Cut the Crap: An Economical Communication Pipeline for LLM-based Multi-Agent Systems (ICLR 2025 · similarity 0.89)
• KVComm: Enabling Efficient LLM Communication through Selective KV Sharing (ICLR 2026 · similarity 0.89)
• HieraMAS: Optimizing Intra-Node LLM Mixtures and Inter-Node Topology for Multi-Agent Systems (ICML 2026 · similarity 0.88)
• Stochastic Self-Organization in Multi-Agent Systems (ICLR 2026 · similarity 0.88)
• SafeSieve: From Heuristics to Experience in Progressive Pruning for LLM-based Multi-Agent Communication (AAAI 2026 · similarity 0.88)