Research map / Multi-agent systems and games / LLM agents

A Two-Tier Perspective on Inference-Time Parallelism in Multi-Agent LLM Systems

ICML 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TIPEX, a two-tier framework separating Replica Parallelism (multiple solution paths with generation and selection) from Structural Parallelism (DAG-based concurrent agent/tool scheduling) on top of Magentic-One, evaluated on GAIA.

Problem

Inference-time parallelism in LLM multi-agent systems is studied only in isolated pieces; the accuracy-latency-cost roles and interactions of task-level and within-trajectory parallelism are unclear.

Approach

Model inference-time parallelism as two decision levels: replica-level exploration with generation (RIG, OHG) and selection (GS, FVES, TKS) strategies, and instance-level dynamic DAG scheduling (SS, BP, AP), unified under one controllable execution framework built on Magentic-One.

Key results

• On GAIA Level 1, OHG+TKS reaches 56.5% accuracy vs 43.2% for Magentic-One, and RIG+GS reaches 56.7%.
• On GAIA Level 2, OHG+GS reaches 38.7% vs 25.8% for Magentic-One, with OHG+TKS at 32.0% and lower latency than GS.
• Level 2 ablation under OHG+TKS shows BP with 5 replicas reaching 37% accuracy vs 29% with one replica, while AP drops accuracy to 25% at 5 replicas.

Surprising findings

• Aggressive structural parallelism (AP) lowers accuracy relative to balanced parallelism while increasing token cost.
• Increasing replicas from 3 to 5 gives marginal or negative accuracy gains in some configurations.

Limitations

• Judge quality is a bottleneck; a persistent gap remains between actual and oracle accuracy.
• Small per-level sample sizes and repeated runs; results come from a single main backbone (Qwen-Plus) plus limited supplementary tests.
• Auto Router is preliminary and does not improve accuracy or token cost.

Experimental setup

Datasets: GAIA GAIA2-mini

Benchmarks: GAIA GAIA2-mini

Metrics: accuracy oracle accuracy wall-clock time token consumption

Baselines: Magentic-One RIG+GS RIG+FVES RIG+TKS OHG+GS OHG+FVES

Builds on: Magentic-One self-consistency LLMCompiler

Closest papers by meaning

• Towards Direct Latent-Space Synthesis for Parallel Branches in LLM-Agent Workflows (NeurIPS 2026 · similarity 0.90)
• AgentTTS: Large Language Model Agent for Test-time Compute-optimal Scaling Strategy in Complex Tasks (NeurIPS 2025 · similarity 0.89)
• Hogwild! Inference: Parallel LLM Generation via Concurrent Attention (NeurIPS 2025 · similarity 0.88)
• When Parallelism Pays Off: Cohesion-Aware Task Partitioning for Multi-Agent Coding (NeurIPS 2026 · similarity 0.88)
• EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective (NeurIPS 2026 · similarity 0.88)
• ThreadWeaver: Adaptive Threading for Efficient Parallel Reasoning in Language Models (ICML 2026 · similarity 0.88)