Research map / Large language models / Retrieval and knowledge
SeCom: On Memory Construction and Retrieval for Personalized Conversational Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SeCom, which builds a conversation memory bank at topical-segment granularity using an LLM-based segmenter and denoises memory units with LLMLingua-2 prompt compression before retrieval, outperforming turn-, session-, and summary-based memory on LOCOMO and Long-MT-Bench+.
Problem
Long-term personalized conversational agents need memory units for retrieval-augmented response generation, but turn-level, session-level, and summary-based memory granularities each suffer from retrieval inaccuracy and loss or noise in retrieved content.
Approach
Construct memory at the topically coherent segment level via a conversation segmentation model (GPT-4 zero-shot, optionally refined by LLM self-reflection on annotated data), and apply LLMLingua-2 compression as a denoising step on memory units before retrieval, without summarization.
Key results
- On LOCOMO with BM25 retriever, SeCom reaches GPT4Score 71.57 versus 65.58 for the best turn-level baseline; with MPNet on Long-MT-Bench+ it reaches 88.81 vs 85.14 for MemoChat.
- Removing compression-based denoising drops LOCOMO GPT4Score from 69.33 to 59.87 with the MPNet retriever.
- Zero-shot segmentation achieves Dialseg711 Pk 0.093 and TIAGE Score 0.607, outperforming unsupervised baselines; reflection-based variant improves further.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use segment-level memory instead of turn-level or session-level memory | SeCom · LOCOMO | GPT4Score: 65.92 -> 71.57 | Helped |
| use segment-level memory instead of turn-level or session-level memory | SeCom · Long-MT-Bench+ | GPT4Score: 85.14 -> 88.81 | Helped |
| add compression-based memory denoising with LLMLingua-2 (ablation) | SeCom with MPNet retrieval · LOCOMO | GPT4Score: 59.87 -> 69.33 | Helped |
| add compression-based memory denoising with LLMLingua-2 (ablation) | SeCom with MPNet retrieval · Long-MT-Bench+ | GPT4Score: 87.51 -> 88.81 | Helped |
| replace GPT-4 segmentation with Mistral-7B segmentation (ablation) | SeCom with MPNet retrieval · LOCOMO | GPT4Score: 69.33 -> 66.37 | Hurt |
| replace GPT-4 segmentation with RoBERTa segmentation (ablation) | SeCom with MPNet retrieval · LOCOMO | GPT4Score: 69.33 -> 61.84 | Hurt |
| replace GPT-4 segmentation with Mistral-7B segmentation (ablation) | SeCom with MPNet retrieval · Long-MT-Bench+ | GPT4Score: 88.81 -> 86.32 | Hurt |
| replace GPT-4 segmentation with RoBERTa segmentation (ablation) | SeCom with MPNet retrieval · Long-MT-Bench+ | GPT4Score: 88.81 -> 81.52 | Hurt |
| use segment-level memory instead of turn-level memory | SeCom · CoQA | GPT4Score: 97.48 -> 98.31 | Helped |
| use segment-level memory instead of turn-level memory | SeCom · Persona-Chat | GPT4Score: 76.83 -> 78.34 | Helped |
| use segment-level memory instead of turn-level memory | SeCom · Long-MT-Bench+ with Mistral-7B-Instruct-v0.3 generation | GPT4Score: 85.61 -> 90.58 | Helped |
| use segment-level memory instead of turn-level memory | SeCom · LOCOMO official QA pairs | GPT4Score: 81.52 -> 84.21 | Helped |
Surprising findings
- Summary-based memory (SumMem, RecurSum) underperforms simple turn-level and session-level memory due to loss of crucial details.
- Turn-level and session-level retrieval performance varies strongly with the retriever (up to ~12 GPT4Score points on LOCOMO), while SeCom is more robust.
Limitations
- Main experiments rely on GPT-4 for segmentation and GPT-3.5-Turbo for generation, which incurs API cost and reliance on proprietary models.
- Long-MT-Bench+ and LOCOMO QA pairs are GPT-4 generated, which may introduce evaluation bias; GPT-4 is also the judge.
- Compression benefit is shown only for compression rates above 50%; the gain depends on the chosen rate (75% used).
Experimental setup
Datasets: LOCOMO Long-MT-Bench+ MT-Bench+ DialSeg711 TIAGE SuperDialSeg CoQA Persona-Chat
Benchmarks: LOCOMO Long-MT-Bench+ DialSeg711 TIAGE SuperDialSeg CoQA Persona-Chat
Metrics: GPT4Score BLEU ROUGE BERTScore pairwise win rate Pk WindowDiff F1 segment score DCG
Baselines: Turn-Level Session-Level Zero History Full History SumMem RecurSum ConditionMem MemoChat COMEDY
Builds on: LLMLingua-2 MemoChat RecurSum prefix-tuning LumberChunker
Closest papers by meaning
- LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (ICLR 2025 · similarity 0.89)
- AMemGym: Interactive Memory Benchmarking for Assistants in Long-Horizon Conversations (ICLR 2026 · similarity 0.88)
- Beyond Prompts: Dynamic Conversational Benchmarking of Large Language Models (NeurIPS 2024 · similarity 0.87)
- MemoryBank: Enhancing Large Language Models with Long-Term Memory (AAAI 2024 · similarity 0.87)
- SimpleMem: Efficient Lifelong Memory for LLM Agents (ICML 2026 · similarity 0.87)
- Personalize-then-Store: Benchmarking and Learning Personalized Memory for Long-Horizon Agents (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.