Research map / Multi-agent systems and games / LLM agents

ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Agentic benchmarks and orchestration · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ReasoningBank, a memory framework that distills generalizable reasoning strategies from both successful and failed agent trajectories (self-judged), and MaTTS, which couples it with parallel/sequential test-time scaling to create a memory-test-time scaling synergy.

Problem

LLM agents deployed over continuous task streams fail to learn from accumulated experience; prior memory methods store raw trajectories or success-only workflows and leave failures underused.

Approach

Distill structured memory items (title, description, content) of transferable reasoning strategies and preventative lessons from both successful and failed trajectories, judged by the agent's own LLM-as-a-judge without ground truth; and scale test-time compute (parallel self-contrast or sequential self-refinement) so that contrastive exploration yields better memory, creating a feedback loop.

Key results

• On WebArena with Gemini-2.5-pro, ReasoningBank raises overall success rate from 46.7 to 53.9 (+7.2) and MaTTS parallel scaling reaches 56.3.
• On Gemini-2.5-flash WebArena-Shopping, MaTTS parallel scaling (k=5) lifts Best-of-5 success from 49.7 to 55.1, versus 52.4 for vanilla TTS.
• On SWE-Bench-Verified with Gemini-2.5-pro, ReasoningBank achieves 57.4 resolve rate vs 54.0 for No Memory, with fewer average steps (19.8 vs 21.1).

Surprising findings

• Scaling test-time compute without good memory reduces Pass@1 for weaker memory methods (Synapse and AWM), suggesting extra rollouts give diminishing returns unless memory is strong.
• Adding failed trajectories degraded AWM (44.4 to 42.2) and only modestly helped Synapse, while ReasoningBank improved with failures included.

Limitations

• Relies on LLM-as-a-judge for success/failure labels, which may be noisy for ambiguous tasks.
• Deliberately simple retrieval and consolidation (no pruning or merging); top-k retrieval beyond k=1 degrades performance.
• Does not compare against other memory architectures such as episodic or hierarchical memory.

Experimental setup

Datasets: WebArena Mind2Web SWE-Bench-Verified

Benchmarks: WebArena Mind2Web SWE-Bench-Verified

Metrics: success rate average steps element accuracy action F1 step success rate resolve rate Best-of-N pass@k

Baselines: No Memory Synapse AWM MaTTS w/o memory MaTTS w/o aggregation (vanilla TTS)

Builds on: ReAct Agent Workflow Memory Synapse self-refinement self-contrast

Closest papers by meaning

• EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective (NeurIPS 2026 · similarity 0.92)
• MemGen: Weaving Generative Latent Memory for Self-Evolving Agents (ICLR 2026 · similarity 0.91)
• AdaMEM: Test-Time Adaptive Memory for Language Agents (ICML 2026 · similarity 0.91)
• MemEvolve: Meta-Evolution of Agent Memory Systems (ICML 2026 · similarity 0.91)
• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.90)
• Structurally Aligned Subtask-Level Memory for Software Engineering Agents (ICML 2026 · similarity 0.90)