Research map / Multi-agent systems and games / LLM agents

Structurally Aligned Subtask-Level Memory for Software Engineering Agents

ICML 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent collaboration and safety · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes subtask-level memory for SWE agents that stores, retrieves, and updates experience per functional subtask (category-filtered, then semantically matched), improving Pass@1 on SWE-bench Verified over vanilla and instance-level memory baselines.

Problem

Instance-level memory for software engineering agents retrieves whole past episodes by global task similarity, which causes misguided retrieval when similar-looking issues need different reasoning at particular stages.

Approach

Segment agent trajectories into functional subtasks (Analyze, Reproduce, Edit, Verify), store abstracted experience keyed by (category, intent description), and retrieve by hard category filter followed by embedding similarity on the current subtask intent, updating memory online after each subtask.

Key results

• Subtask-level memory improves average Pass@1 over vanilla by +4.7 pp across four backbones; +6.8 pp on Gemini 2.5 Pro (53.5 to 60.3).
• On Claude 3.7 Sonnet, instance-level memory drops to 51.1 from 52.2 vanilla while subtask-level memory reaches 56.1.
• On Hard instances (>28 steps), success rises from 35.5% to 44.2% (+8.7 pp) with subtask memory.

Surprising findings

• Structured prompting alone yields only +1.0 pp over vanilla, so most gain comes from the injected experience rather than the workflow structure.
• Instance-level memory degrades Claude 3.7 Sonnet performance (52.2 to 51.1 average Pass@1).

Limitations

• Evaluated only on SWE-bench Verified with one scaffold (Mini SWE Agent).
• Memory extraction uses the same backbone, so no stronger teacher is assessed; ablations run on a single backbone (Claude 3.7 Sonnet).
• Improvements on Claude 4.0 Sonnet are modest (+2.3 pp).

Experimental setup

Datasets: SWE-bench Verified

Benchmarks: SWE-bench Verified

Metrics: Pass@1 Best@3 mean and std over 3 runs

Baselines: Vanilla Mini SWE Agent ReasoningBank (instance-level memory)

Builds on: ReasoningBank SWE-bench Mini SWE Agent

Closest papers by meaning

• ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory (ICLR 2026 · similarity 0.90)
• EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective (NeurIPS 2026 · similarity 0.90)
• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.90)
• AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications (ICML 2026 · similarity 0.89)
• M$^\star$: Every Task Deserves Its Own Memory Harness (NeurIPS 2026 · similarity 0.88)
• A-Mem: Agentic Memory for LLM Agents (NeurIPS 2025 · similarity 0.88)