Research map / Multi-agent systems and games / LLM agents

Narrowing the Collaboration Gap, Probably

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows theoretically and empirically that two agents communicating calibrated probability distributions over actions outperform communicating only actions on chained-parity and collaborative maze tasks, and that post-hoc conversational calibration repairs biased probability messages for both trained transformers and pretrained LLMs.

Problem

How should multiple agents with different private observations communicate concisely so that decision-relevant information is aggregated; action-only messages discard uncertainty that matters for combining information.

Approach

Agents communicate calibrated beliefs over actions instead of only proposed actions; a chained-parity model proves probability exchange transmits one new bit per round while action exchange stalls, and a post-hoc calibrator (trained on labelled transcripts using the current and previous messages) corrects biased reports so they remain conversation-calibrated.

Key results

• On chained parities, learned probability messages track the theoretical optimum while learned action messages stall after the first bit (Figure 3).
• For GPT-5.4-nano, GPT-5.4-mini and GPT-5, verbalized probability exchange performs similar to or worse than action exchange; post-hoc calibration at least doubles mean maze success after four rounds relative to base verbalized probabilities.
• In the transformer experiments, conversation calibration recovers nearly all of the unbiased baseline success when both global miscalibration and sycophantic anchoring are present.

Surprising findings

• Unrestricted full-response communication performs close to zero success, so sharing more information can hurt collaboration.
• Raw verbalized probabilities are often no better than action-only messages for LLM agents.

Limitations

• Maze experiments use small 6x6 mazes with a single correct path and restricted settings.
• The calibrator cannot add maze-specific knowledge absent from communicated probabilities, and LLMs were not trained on this task.
• Chained-parity model is a stylized abstraction of general reasoning.

Experimental setup

Datasets: synthetic chained-parity bitstrings (2M examples) synthetic 6x6 partial-view maze dataset (2M examples)

Benchmarks: chained-parity (synthetic) collaborative 6x6 maze solving (Davidson et al. 2025 setup)

Metrics: maze success rate squared loss conversation calibration loss

Baselines: action-only communication verbalized probabilities sampled probabilities (M=10) unrestricted full-response communication naive rare-action correction self-calibration full-information solo

Builds on: Collina et al. conversation calibration Davidson et al. collaborative maze benchmark Frongillo et al. agreement NanoGPT

Closest papers by meaning

• Improving Cooperation in Language Games with Bayesian Inference and the Cognitive Hierarchy (AAAI 2025 · similarity 0.89)
• Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs (ICML 2026 · similarity 0.88)
• Position: Uncertainty Quantification Needs Reassessment for Large Language Model Agents (ICML 2025 · similarity 0.88)
• Semantics and Spatiality of Emergent Communication (NeurIPS 2024 · similarity 0.87)
• CoDe: Communication Delay-Tolerant Multi-Agent Collaboration via Dual Alignment of Intent and Timeliness (AAAI 2025 · similarity 0.87)
• From Assumptions to Actions: Turning LLM Reasoning into Uncertainty-Aware Planning for Embodied Agents (ICLR 2026 · similarity 0.87)