Research map / Multi-agent systems and games / LLM agents

EvoTest: Evolutionary Test-Time Learning for Self-Improving Agentic Systems

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces the J-TTL benchmark (repeated Jericho text-game episodes) and EvoTest, a gradient-free framework where an Evolver agent rewrites the prompt, memory, hyperparameters and tool routines of an Actor agent between episodes, selected by UCB.

Problem

Current LLM agents cannot learn complex skills on the fly across repeated attempts at the same task at test time; existing reflection, memory, prompt-optimization and online fine-tuning methods struggle in sparse-reward interactive fiction games.

Approach

Evolve the entire agentic configuration (prompt, structured memory, hyperparameters, tool-use routines) between episodes using an Evolver LLM that analyzes the full episode transcript, with UCB selection among parent and child configurations, requiring no gradients.

Key results

• EvoTest achieves average AUC 0.47 (Gemini-2.5-flash) and 0.50 (Claude-4-sonnet) across six games, versus 0.34/0.36 for the best prompt-evolution baseline (EvoPrompt) and 0.30 for online GRPO.
• EvoTest is the only method reported to win two games (Detective and Library), while all baselines win none.
• With qwen3-32b as actor and o3 as evolver, EvoTest reaches average AUC 0.40 vs 0.31 for online GRPO on the same backbone.

Surprising findings

• Online GRPO and SFT underperform many prompt-based and reflection-based methods, with GRPO scoring below Reflexion on average despite weight updates.
• A simple generic prompt-mutation instruction drops Detective AUC from 0.94 to 0.65, showing that the structure of the Evolver prompt matters greatly.

Limitations

• Performance is capped by the frozen backbone's latent capabilities; no weight updates to learn new skills.
• Learned strategies are task-specific and may not generalize to variations of the game.
• Depends on a strong Evolver LLM; simple (1+m) evolution with UCB may converge to local optima.

Experimental setup

Datasets: Jericho Detective Jericho Library Jericho Zork1 Jericho Zork3 Jericho Balances Jericho Temple

Benchmarks: J-TTL (Jericho Detective, Library, Zork1, Zork3, Balances, Temple)

Metrics: AUC of per-episode score over 50 episodes final episode score

Baselines: Static Memory RAG Summary Reflexion TextGrad Promptbreeder EvoPrompt SFT (online) GRPO (online)

Builds on: Reflexion TextGrad Promptbreeder EvoPrompt GRPO Jericho

Closest papers by meaning

• AdaMEM: Test-Time Adaptive Memory for Language Agents (ICML 2026 · similarity 0.88)
• MemEvolve: Meta-Evolution of Agent Memory Systems (ICML 2026 · similarity 0.88)
• MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory (NeurIPS 2026 · similarity 0.88)
• ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory (ICLR 2026 · similarity 0.88)
• EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective (NeurIPS 2026 · similarity 0.88)
• PACE: Two-Timescale Self-Evolution for Small Language Model Agents (NeurIPS 2026 · similarity 0.87)