Research map / Multi-agent systems and games / LLM agents
Dual Latent Memory for Visual Multi-agent System
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes L²-VMAS, a model-agnostic framework that replaces text-based inter-agent communication in visual multi-agent systems with decoupled perception and thinking latent memories, synthesized across turns and retrieved via entropy-triggered proactive orchestration.
Problem
Visual multi-agent systems show a "scaling wall": more agent turns degrade accuracy while token costs grow sharply, attributed to lossy, conflated text-based inter-agent communication of perception and thinking.
Approach
Shares a dual latent memory (separate perception and thinking memory banks of hidden states) across agents, with compressors that synthesize and merge memory units, and an entropy-triggered gate that injects the relevant memory only on demand, while keeping all VLM backbones frozen.
Key results
- On average across five backbones, L²-VMAS improves accuracy by 2.7-5.4% over VMAS while reducing token usage by 21.3-44.8%.
- On Qwen3-VL-8B-Thinking, L²-VMAS reaches 76.5 average accuracy vs 72.5 for VMAS and cuts tokens from 6154 to 3473 (-43.6%).
- Across multi-image and video benchmarks on Qwen3-VL-8B, average gains are 4.0% (Instruct) and 4.6% (Thinking) over VMAS.
Surprising findings
- Standard text-based VMAS accuracy peaks at about the 3rd agent turn and falls below the single-agent baseline by the 6th turn, while token use grows over 30x by turn 10.
- Transmitting only conclusions outperforms transmitting full content in later turns, indicating that unstructured perception and thinking conflation hurts performance.
Limitations
- Some cells show small accuracy drops (e.g., LLaVA-OV-1.5-8B on MMBench), attributed to benchmark saturation.
- Gains are smaller on simpler benchmarks and smaller models, per the authors' own analysis.
Experimental setup
Datasets: GQA
Benchmarks: MMBench MMStar RealWorldQA SimpleVQA MuirBench BLINK MVBench LVBench
Metrics: accuracy total token usage
Baselines: Single agent VMAS (text-based multi-agent system)
Builds on: PPO Gumbel-Sigmoid
Closest papers by meaning
- Latent Collaboration in Multi-Agent Systems (ICML 2026 · similarity 0.89)
- G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems (NeurIPS 2025 · similarity 0.87)
- Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.87)
- Scaling Large Language Model-based Multi-Agent Collaboration (ICLR 2025 · similarity 0.85)
- Visual Multi-Agent System: Mitigating Hallucination Snowballing via Visual Flow (ICLR 2026 · similarity 0.85)
- A-Mem: Agentic Memory for LLM Agents (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.