Research map / Multi-agent systems and games / LLM agents

UMEM: Unified Memory Extraction and Management Framework for Generalizable Memory

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Agent data synthesis and behavior studies · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes UMEM, which jointly trains an LLM Mem-Optimizer to extract and manage self-evolving memories using GRPO with a neighborhood-level marginal utility reward and online memory evolution.

Problem

Self-evolving LLM agents accumulate instance-specific noise in memory because memory extraction is treated as a static process while only management is optimized, leading to poor generalization and misalignment between extracted memories and management policy.

Approach

Jointly optimize memory extraction and management with GRPO, using Semantic Neighborhood Modeling (top-N semantically similar queries) to compute a marginal utility reward (success gain plus gated efficiency regularization) averaged over the neighborhood, so memories are judged by reuse across related queries.

Key results

• UMEM-Qwen3-4B with GPT-5.1 executor reaches 82.84% ALFWorld success rate versus 70.90% for the no-train Qwen3-4B baseline and 77.61% for Memp; average 57.74 vs 54.71 for Memp.
• Average gain of UMEM-Qwen3-4B over no-memory backbone is +3.9 points with Qwen3-8B-Thinking executor and +4.6 with GPT-5.1.
• Ablation: removing extraction optimization drops average by about 4.7 points, while removing management optimization drops about 0.73 points (GPT-5.1 executor).

Surprising findings

• Optimizing memory management alone (ReMem) degrades fastest under continual streaming, while extraction optimization matters more than management optimization in ablations.

Limitations

• Training data limited to MMLU-derived queries; generalization evaluated on a small set of benchmarks.
• Reported ablations and main results come from a single run with small benchmark sizes (e.g., AIME 60 questions) so differences may be noisy.
• Memory actions restricted to ADD and UPDATE in the main method; no DELETE explored.

Experimental setup

Datasets: MMLU

Benchmarks: AIME GPQA-Diamond HLE HotpotQA ALFWorld

Metrics: exact match accuracy cumulative success rate progress rate average steps

Baselines: No Memory Self-RAG Memp ReMem Llama-3.2-1B-Instruct Qwen3-4B-Instruct

Builds on: GRPO ReMem Memp Self-RAG BGE-M3

Closest papers by meaning

• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.92)
• A-Mem: Agentic Memory for LLM Agents (NeurIPS 2025 · similarity 0.91)
• EvoMemBench: Benchmarking Agent Memory from a Self-Evolving Perspective (NeurIPS 2026 · similarity 0.90)
• PlugMem: A Task-Agnostic Plugin Memory Module for LLM Agents (ICML 2026 · similarity 0.89)
• AMemGym: Interactive Memory Benchmarking for Assistants in Long-Horizon Conversations (ICLR 2026 · similarity 0.89)
• AdaMEM: Test-Time Adaptive Memory for Language Agents (ICML 2026 · similarity 0.89)