Research map / Multi-agent systems and games / LLM agents

FlashEvolve: Accelerating Agent Self-Evolution with Asynchronous Stage Orchestration

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Robust and efficient agent execution · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes FlashEvolve, an asynchronous worker-and-queue framework for LLM-based agent self-evolution that overlaps stages, handles artifact staleness via version tracking and reflective patching, and reports 3.5x (vLLM) and 4.9x (API) proposal throughput gains over synchronous GEPA.

Problem

Agent self-evolution (refining prompts, memory, harness code, or programs with LLMs) is slow in wall-clock time due to synchronized sequential stages and long-tail generation imbalance within each LLM-heavy stage.

Approach

Replace synchronized stage execution with asynchronous workers connected by queues, tag each item with artifact-pool version, and handle staleness with update/discard/reflective-patch policies; add speculative stage completion and adaptive worker reallocation.

Key results

• On GEPA workloads with local vLLM Qwen3-8B, FlashEvolve improves proposal throughput by 3.5x on average over GEPA; on API GPT-4o-mini serving it improves by 4.9x.
• Within a 30-minute budget on IFBench, FlashEvolve raises validation score from 87.6 to 90.6 with a 2.27x normalized evolution rate.
• On AIME, FlashEvolve reaches 15.0% validation score while GEPA and Combee stay at the initial 10.0%.

Surprising findings

• Naively increasing worker counts shifts the bottleneck across stages and does not scale validation throughput uniformly.
• Speculative completion helps only with a properly set prefix threshold; alpha=0.5 reduced validation throughput, so the authors treat it as optional.

Limitations

• Each algorithm integration requires algorithm-specific implementation effort.
• Evaluation focuses on prompt, context, and harness-code evolution workloads; coverage of algorithms and artifact types is limited.
• Open-source model's weak code-generation capability slows Meta-Harness evolution progress.

Experimental setup

Datasets: IFBench HotpotQA HoVer AIME FiNER FormulaReasoning Symptom2Disease AGNews

Benchmarks: IFBench HotpotQA HoVer AIME FiNER FormulaReasoning Symptom2Disease AGNews

Metrics: LLM throughput (token/s) proposal throughput (proposal/min) validation score normalized evolution rate

Baselines: GEPA Combee (K=10) Combee (K=40)

Builds on: GEPA ACE Meta-Harness Combee vLLM

Closest papers by meaning

• PACE: Two-Timescale Self-Evolution for Small Language Model Agents (NeurIPS 2026 · similarity 0.87)
• Evaluating Test-Time Scaling of General LLM Agents (NeurIPS 2026 · similarity 0.85)
• MemEvolve: Meta-Evolution of Agent Memory Systems (ICML 2026 · similarity 0.85)
• ScaleSim: Serving Large-Scale Multi-Agent Simulation with Invocation Distance-Based Memory Management (ICML 2026 · similarity 0.85)
• EvoCodeBench: Evaluating Coding Agents in Multi-Turn Iterative Interactions (NeurIPS 2026 · similarity 0.85)
• Your Agent May Misevolve: Emergent Risks in Self-evolving LLM Agents (ICLR 2026 · similarity 0.85)