Research map / Multi-agent systems and games / LLM agents
Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-Horizon Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PerMem-Bench, a multi-domain long-horizon benchmark with user-specific memory necessity profiles, and studies session-level storage gating (Greedy, Context-aware, Structure-aware) for personalized memory in Mem0, Memory-R1, and RMM.
Problem
Existing LLM memory systems apply universal storage policies to all users, wasting memory budget on transient sessions and dropping context needed for long-horizon tasks; no benchmark captures user-specific agent use patterns across domains.
Approach
Treat storage as personalized: a session-level gate predicts whether each session is long-horizon or transient and skips memory operations for transient ones, wrapping existing memory systems without modifying them; a fully automated persona-to-life-skeleton-to-dialogue pipeline builds benchmarks with per-user memory-necessity profiles.
Key results
- Structure-aware gating reaches up to 0.844 F1 on PerMem-Bench-s, while Greedy and Context-aware gating are weaker.
- Oracle personalization yields substantial retention gains over Universal across Mem0, Memory-R1, and RMM, with the largest gap at budgets of 100 and 200 entries.
- Greedy and Context-aware gating consistently underperform Universal, and Structure-aware gives only marginal gains despite higher gating accuracy.
Surprising findings
- Structure-aware gating collapses right after a long-horizon-to-transient profile shift because it over-relies on established project clusters, while Greedy shows no such collapse.
- Imperfect gating can make personalization worse than universal storage, with Greedy and Context-aware underperforming Universal.
Limitations
- Only 20 users and a simple two-dimensional profile (domain participation and memory necessity).
- Gating only affects storage; no personalized deletion or retroactive correction.
- Retention metric relies on an LLM judge (gpt-5-nano) and checkpoint sampling approximation.
Experimental setup
Datasets: Nemotron-Persona-USA
Benchmarks: PerMem-Bench
Metrics: memory retention rate F1 false negative rate false positive rate
Baselines: Universal (no gating) Oracle gating Greedy gating Context-aware gating Structure-aware gating
Builds on: Mem0 Memory-R1 RMM LoCoMo
Closest papers by meaning
- AMemGym: Interactive Memory Benchmarking for Assistants in Long-Horizon Conversations (ICLR 2026 · similarity 0.92)
- How Does Personalized Memory Shape LLM Behavior? Benchmarking Rational Preference Utilization in Personalized Assistants (NeurIPS 2026 · similarity 0.90)
- Embodied Agents Meet Personalization: Investigating Challenges and Solutions Through the Lens of Memory Utilization (ICLR 2026 · similarity 0.90)
- Mem-PAL: Towards Memory-based Personalized Dialogue Assistants for Long-term User-Agent Interaction (AAAI 2026 · similarity 0.89)
- Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.89)
- A-Mem: Agentic Memory for LLM Agents (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.