Research map / Large language models / Retrieval and knowledge

Beyond RAG for Agent Memory: Retrieval by Decoupling and Aggregation

NeurIPS 2026 · Large language models › Retrieval and knowledge · approach: RAG robustness and evaluation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes xMemory, a revisable hierarchical agent-memory structure (segments → decoupled memory components → groups) with top-down retrieval that selects a coverage-aware backbone and expands to segments/messages only when uncertainty drops, evaluated on LoCoMo and PerLTQA.

Problem

Flat top-k similarity retrieval over agent memory returns redundant, near-duplicate context, while summary-centric hierarchies blur the fine details needed to distinguish candidate evidence.

Approach

Apply "decoupling before aggregation": extract reusable memory components from local segments first, then aggregate components into groups via a sparsity–semantic faithfulness objective that is maintained incrementally (split/merge), and retrieve top-down with a coverage-greedy backbone followed by uncertainty-guided expansion.

Key results

• On LoCoMo with Qwen3-8B, xMemory improves average BLEU/F1 from 31.44/40.88 (Nemori) to 34.48/43.98 while using 4711 tokens per query versus 7755 for Nemori.
• With GPT-5 nano on LoCoMo, average BLEU/F1 rises from 36.65/48.17 (Nemori) to 38.71/50.00 with 6581 tokens per query.
• On PerLTQA with Qwen3-8B, xMemory achieves BLEU/F1/ROUGE-L of 36.24/47.08/42.50 with the fewest tokens (5087).

Surprising findings

• Pruning-based RAG (LLMLingua-2) reduces tokens relative to Naive RAG but lowers accuracy and requires more blocks to cover evidence, indicating fragmentation of evidence.

Limitations

• Uncertainty for closed-source readers (GPT-5 nano) is estimated with a proxy model (GPT-4.1-mini), which is an approximation.
• Evaluated only on two benchmarks (LoCoMo and PerLTQA) with lexical metrics (BLEU, F1, ROUGE-L) and no LLM-judge or human evaluation.
• Fano-based motivation for group size is a loose argument; the chosen group cap of 12 is selected empirically.

Experimental setup

Datasets: LoCoMo PerLTQA

Benchmarks: LoCoMo PerLTQA

Metrics: BLEU-1 F1 ROUGE-L tokens per query

Baselines: Full Memory Naive RAG LightMem Nemori A-Mem MemoryOS

Builds on: retrieval-augmented generation Fano's inequality

Closest papers by meaning

• A-Mem: Agentic Memory for LLM Agents (NeurIPS 2025 · similarity 0.89)
• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.89)
• Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agents (ICML 2026 · similarity 0.89)
• Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction (NeurIPS 2026 · similarity 0.89)
• SimpleMem: Efficient Lifelong Memory for LLM Agents (ICML 2026 · similarity 0.89)
• AMA-Bench: Evaluating Long-Horizon Memory for Agentic Applications (ICML 2026 · similarity 0.89)