Research map / Large language models / Retrieval and knowledge
Augmenting Language Models with Long-Term Memory
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LongMem’s decoupled long-term memory with a frozen backbone and trainable SideNet instead of GPT-2* without memory | GPT-2* · PG-22 length splits | perplexity: -1.38 to -1.62 perplexity | Helped |
| Use LongMem’s decoupled memory rather than MemTRM memory augmentation | GPT-2* · ArXiv | perplexity: -1.0 ppl | Helped |
| Load long prefix context into LongMem memory rather than use MemTRM | GPT-2* · ChapterBreak AO3 | accuracy: 28.3% -> 37.7%; 28.7% -> 39.4%; 28.7% -> 40.5% | Helped |
| Add 200 in-memory demonstrations to LongMem for SQuAD 3-shot | LongMem · SQuAD 3-shot | EM: +4.5 EM score | Helped |
| Add 2000 in-memory demonstrations using LongMem | LongMem · NLU five-task average, 4-shot | accuracy: 55.6 -> 58.4 | Helped |
| Add 2000 in-memory demonstrations using LongMem | LongMem · NLU five-task average, 20-shot | accuracy: 58.6 -> 66.7 | Helped |
| Use chunk-size 2 rather than chunk-size 4 or 8 (ablation) | LongMem · Five NLU datasets | accuracy | Helped |
| Set inference memory size to 16k rather than 8k, 32k, or 65k (ablation) | LongMem · PG-22 language modeling | perplexity | Helped |
Closest papers by meaning
- M+: Extending MemoryLLM with Scalable Long-Term Memory (ICML 2025 · similarity 0.91)
- A Little Goes a Long Way: Efficient Long Context Training and Inference with Partial Contexts (ICLR 2025 · similarity 0.91)
- Retrieval meets Long Context Large Language Models (ICLR 2024 · similarity 0.90)
- SCBench: A KV Cache-Centric Analysis of Long-Context Methods (ICLR 2025 · similarity 0.90)
- DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads (ICLR 2025 · similarity 0.89)
- Memory Decoder: A Pretrained, Plug-and-Play Memory for Large Language Models (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.