Research map / Continual learning / Online and replay-based
Learning to Continually Learn via Meta-learning Agentic Memory Designs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ALMA, which uses a Meta Agent (GPT-5) to open-endedly search code-based agentic memory designs, finding designs that beat hand-crafted memory baselines on four sequential decision-making benchmarks.
Problem
Agentic systems built on stateless foundation models need memory to continually learn from experience, but memory designs are hand-crafted and fixed, limiting adaptation across diverse and non-stationary tasks.
Approach
Treat memory design as executable Python code and use a Meta Agent to iteratively ideate, implement, and evaluate new memory designs from an open-ended archive (DGM-style sampling), rather than greedy selection from handcrafted initializations.
Key results
- Learned memory design reaches 12.3% overall average success with GPT-5-nano vs 7.7% for best human-designed baseline (G-Memory) and 6.1% no-memory.
- With GPT-5-mini as agent, learned design averages 53.9% vs 48.6% for Trajectory Retrieval and 41.1% no-memory (+12.8 points).
- On ALFWorld distribution shift with GPT-5-mini, learned design reaches 84.1% success, above all human-designed baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Learned memory design via open-ended meta-learning (ALMA) vs. hand-designed memory baselines | agentic system with GPT-5-nano · ALFWorld, TextWorld, Baba Is AI, MiniHack | overall success rate: 12.3 vs 8.6 best baseline (Trajectory Retrieval) | Helped |
| Learned memory design transferred to stronger GPT-5-mini agent | agentic system with GPT-5-mini · overall across four benchmarks | overall success rate: 53.9 vs 48.6 best baseline (Trajectory Retrieval) | Helped |
| Greedy-selection search instead of open-ended exploration for memory design learning (ablation) | ALMA meta-learning loop · ALFWorld, GPT-5-nano and GPT-5-mini testing | success rate: 11.9 vs 12.4 (nano); 77.1 vs 87.1 (mini) | Hurt |
| Memory collected from more trajectories (task size during collection) | learned vs manual memory designs · ALFWorld, GPT-5-mini | success rate | Mixed |
| Learned memory design under distribution shift (valid_seen collection, valid_unseen dynamic deployment) | ALMA vs human-designed memory · ALFWorld valid_unseen, GPT-5-mini | success rate: 84.1% for ALMA, outperforming all baselines | Helped |
Surprising findings
- Greedy search over designs yields lower success (11.9% vs 12.4% on ALFWorld with GPT-5-nano), supporting open-ended exploration over greedy selection.
- Increasing memory collection size does not necessarily improve success monotonically, which the authors attribute to trajectory quality variance.
Limitations
- Memory designs are learned on a predefined learning set rather than online during deployment.
- Search is restricted to code-space memory on token-level memory; does not co-learn the agentic system or model weights.
- Computational budget limits: only 11 learning steps and 43 designs explored; safety relies on sandboxing and human oversight.
Experimental setup
Datasets: ALFWorld TextWorld Baba Is AI MiniHack BALROG
Benchmarks: ALFWorld TextWorld Baba Is AI MiniHack
Metrics: success rate end-to-end memory cost retrieved token size
Baselines: Trajectory Retrieval ReasoningBank Dynamic Cheatsheet G-Memory greedy search ablation
Builds on: ADAS Darwin Gödel Machine ReasoningBank Dynamic Cheatsheet G-Memory
Closest papers by meaning
- MemEvolve: Meta-Evolution of Agent Memory Systems (ICML 2026 · similarity 0.90)
- AdaMEM: Test-Time Adaptive Memory for Language Agents (ICML 2026 · similarity 0.89)
- MemGen: Weaving Generative Latent Memory for Self-Evolving Agents (ICLR 2026 · similarity 0.89)
- MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents (NeurIPS 2026 · similarity 0.89)
- Automated Design of Agentic Systems (ICLR 2025 · similarity 0.88)
- MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.