Research map / Multi-agent systems and games / LLM agents

Towards Direct Latent-Space Synthesis for Parallel Branches in LLM-Agent Workflows

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Parallel-Synthesis, a plug-and-play framework where a synthesizer LLM directly consumes re-encoded KV caches of parallel worker branches via a learned cache mapper and LoRA, avoiding text re-prefill.

Problem

Synthesizing outputs of parallel agent branches with LLMs requires concatenating their text and re-prefilling it, which discards DAG structure and adds redundant prefill cost; existing cache-reuse methods from RAG are poorly suited to this many-to-one agent setting.

Approach

Reuse worker KV caches directly at synthesis time after positional re-encoding (aligning all branches to a shared post-branch position range), calibrate them with a learned worker-conditioned affine cache mapper, and post-train a synthesizer LoRA via two data tracks (general parallel-context adaptation and distillation from text-serialization synthesis), then merge checkpoints.

Key results

• Parallel-Synthesis matches or outperforms Text-Serialization on 7 of 9 datasets, e.g., AIME 2025 46.67 vs 23.33 and GPQA 52.02 vs 50.00.
• Reduces time-to-first-token by 2.5x to 11x relative to Text-Serialization on five datasets.
• Outperforms inference-time cache-reuse baselines APE (e.g., GSM8K 49.66) and CacheBlend (e.g., GSM8K 79.15) by large margins, while KVLINK is the strongest cache baseline.

Surprising findings

• Parallel-Synthesis beats text-serialization on several reasoning-heavy tasks despite not seeing the text worker outputs.
• Sequential Track 1 then Track 2 tuning underperforms both single tracks and the merged checkpoint.

Limitations

• Gains are not uniform: small gaps on two of nine datasets and weaker on evidence-centric tasks such as MBPP-Plus and some GAIA cases.
• Evaluation on small sample sizes (e.g., 30 AIME problems, 53 GAIA Level 1 items), so differences may be noisy.
• Not yet suitable to replace text communication in highly complex structured workflows, per authors.

Experimental setup

Datasets: WildChat UltraChat LMSYS-Chat Toucan DTA-Tool FLAN 2WikiMultiHopQA BrowseComp AIME 2024 AIME 2025 GSM8K GPQA

Benchmarks: AIME 2024 AIME 2025 GSM8K HumanEvalPlus MBPPPlus GPQA MedQA GAIA MARBLE Database

Metrics: accuracy time-to-first-token

Baselines: Single Voting Text-Serialization CacheBlend APE KVLINK

Builds on: APE CacheBlend KVLINK RoPE

Closest papers by meaning

• KVFlow: Efficient Prefix Caching for Accelerating LLM-Based Multi-Agent Workflows (NeurIPS 2025 · similarity 0.90)
• A Two-Tier Perspective on Inference-Time Parallelism in Multi-Agent LLM Systems (ICML 2026 · similarity 0.90)
• KVCOMM: Online Cross-context KV-cache Communication for Efficient LLM-based Multi-agent Systems (NeurIPS 2025 · similarity 0.89)
• Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agents (NeurIPS 2025 · similarity 0.88)
• AgentSwift: Efficient LLM Agent Design via Value-Guided Hierarchical Search (AAAI 2026 · similarity 0.88)
• When Parallelism Pays Off: Cohesion-Aware Task Partitioning for Multi-Agent Coding (NeurIPS 2026 · similarity 0.88)