Research map / Multi-agent systems and games / LLM agents

MATA: A Trainable Hierarchical Automaton System for Multi-Agent Visual Reasoning

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MATA, a hierarchical finite-state automaton for visual reasoning whose top-level transitions are chosen by an SFT-trained LLM hyper agent over three agents, trained on a 90K transition-trajectory dataset.

Problem

Single-agent or hand-crafted compositional visual reasoning pipelines cannot decide when to collaborate across complementary agents or compete among overlapping ones, and hand-written transition rules scale poorly.

Approach

A hierarchical automaton whose states are agents, each with a rule-based sub-automaton, and a learned hyper agent (LLM) that picks the next agent from shared memory; transition labels come from expanding transition-trajectory trees scored by task metrics and distilled via SFT.

Key results

• MATA reaches 64.9% on GQA, above HYDRA (52.8%) and InternVL3.5 (63.8%) and Qwen2.5-VL (62.4%).
• MATA reaches 76.5% on OK-VQA versus 75.7% for InternVL3.5 and 59.4% for HYDRA.
• MATA achieves 96.3 / 93.9 / 90.8 on RefCOCO / RefCOCO+ / RefCOCOg and 77.3 on Ref-Adv, state of the art among compared methods.

Surprising findings

• Swapping the Qwen3-4B LLM state controller for the Qwen3-VL-4B VLM yields near-identical accuracy, so the controller is insensitive to backbone type.
• Direct answer-only SFT of a VLM often hurts cross-task generalization (e.g., InternVL2.5 drops to 35.8 on GQA after OK-VQA SFT).

Limitations

• Training data generation uses near-exhaustive transition tree expansion, which grows rapidly as agents/states are added.
• Only three agents are used; gains diminish from 2 to 3 agents on current benchmarks.
• Ref-Adv has no training split, and results rely on the hyper agent generalizing from other datasets.

Experimental setup

Datasets: GQA OK-VQA RefCOCO RefCOCO+ RefCOCOg

Benchmarks: GQA OK-VQA RefCOCO RefCOCO+ RefCOCOg Ref-Adv

Metrics: accuracy IoU-based accuracy inference time

Baselines: InternVL3.5 Qwen2.5-VL Qwen3-VL HYDRA VisRep ViperGPT DWIM NAVER IdealGPT GPT-4o

Builds on: ViperGPT HYDRA NAVER InternVL2.5 Florence2

Closest papers by meaning

• Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models (CVPR 2025 · similarity 0.90)
• Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks (ICLR 2026 · similarity 0.90)
• Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoning (ICML 2026 · similarity 0.89)
• Train the Agent, Not the Expert: Learning to Harness Heterogeneous Experts for Multi-Turn Visual Reasoning (NeurIPS 2026 · similarity 0.89)
• ATLAS: Agentic or Latent Visual Reasoning? One Word is Enough for Both (NeurIPS 2026 · similarity 0.89)
• GAM-Agent: Game-Theoretic and Uncertainty-Aware Collaboration for Complex Visual Reasoning (NeurIPS 2025 · similarity 0.89)