Research map / Multi-agent systems and games / LLM agents

Mem-T: Densifying Rewards for Long-Horizon Memory Agents

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Mem-T, a hierarchical memory agent trained with MoT-GRPO, a tree-guided RL method that backpropagates node-wise rewards and hindsight-credits memory construction, evaluated on LoCoMo, HotpotQA, LongMemEval and NarrativeQA.

Problem

Memory agents trained with RL receive sparse, delayed terminal rewards over long sequences of memory operations, making credit assignment for memory construction and retrieval hard.

Approach

Builds multiple memory operation trees per query by branching rollouts, assigns dense node-wise rewards via backpropagated averages of leaf F1, uses dual-scale (intra- and inter-tree) advantages for retrieval, and propagates leaf advantages back to source memory operations via hindsight credit assignment for construction.

Key results

• On LoCoMo with Qwen3-4B, Mem-T with MoT-GRPO reaches 58.65 overall F1, a 14.92 F1 gain over the best reported Mem0-style baseline and 5.09 over GRPO-only training.
• With Qwen3-8B, Mem-T achieves 58.53 overall F1 on LoCoMo, improving on training-free and Memory-R1 baselines.
• Out-of-domain, Mem-T averages 54.15 across HotpotQA, LongMemEval and NarrativeQA versus 47.70 for the best baseline GAM.

Surprising findings

• Removing retrieval optimization causes the largest ablation drop (5.28 F1), larger than removing construction optimization (3.29).
• GAM's F1 drops by 23.31 when its backbone is switched from gpt-4o-mini to Qwen3-4B, indicating heavy dependence on the base model.

Limitations

• Construction training uses a small set of 10k memory operations and an offline SFT-style objective rather than full online RL.
• Ground-truth evidence is needed for the evidence alignment gate; without it the method relies only on the retrieval trace gate.
• Inference cost grows linearly with retrieval steps; token use per query is ~9k-21k.

Experimental setup

Datasets: LoCoMo HotpotQA LongMemEval NarrativeQA

Benchmarks: LoCoMo HotpotQA LongMemEval NarrativeQA

Metrics: F1 BLEU-1 Accuracy inference tokens per query

Baselines: Vanilla RAG MemGPT MemoryBank Mem0 MemoryOS LightMem A-Mem GAM MEM1 MemAgent Memory-R1

Builds on: GRPO Memory-R1 A-Mem Mem0 GAM

Closest papers by meaning

• MEM1: Learning to Synergize Memory and Reasoning for Efficient Long-Horizon Agents (ICLR 2026 · similarity 0.91)
• MemSkill: Learning and Evolving Memory Skills for Self-Evolving Agents (NeurIPS 2026 · similarity 0.90)
• Stable Hadamard Memory: Revitalizing Memory-Augmented Agents for Reinforcement Learning (ICLR 2025 · similarity 0.90)
• Learning Query-Aware Budget-Tier Routing for Runtime Agent Memory (ICML 2026 · similarity 0.89)
• Meta-Cognitive Memory Policy Optimization for Long-Horizon LLM Agents (NeurIPS 2026 · similarity 0.89)
• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.89)