Research map / Large language models / Retrieval and knowledge
Beyond a Million Tokens: Benchmarking and Enhancing Long-Term Memory in LLMs
Mohammad Tavakoli, Alireza Salemi, Carrie Ye, Mohamed Abdalla, Hamed Zamani, J Ross Mitchell
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LIGHT (episodic retrieval + scratchpad + working memory) over best baseline | Llama-4-Maverick · BEAM 100K | average nugget score: 0.323 -> 0.358 | Improved |
| use LIGHT over RAG | Llama-4-Maverick · BEAM 10M | average nugget score: 0.249 -> 0.266 | Improved |
| use LIGHT over long-context vanilla | GPT-4.1-nano · BEAM 1M | average nugget score: 0.191 -> 0.336 | Improved |
| use LIGHT over RAG | Gemini-2.0-flash · BEAM 10M | average nugget score: 0.216 -> 0.192 | Hurt |
| remove retrieval (episodic memory) from LIGHT (ablation) | LIGHT · BEAM 100K | performance: 0% | No clear effect |
| add scratchpad to LIGHT (ablation) | LIGHT · BEAM 100K | performance: -1.1% when removed | Improved |
| add noise filtering of scratchpad to LIGHT (ablation) | LIGHT · BEAM 100K | performance: -2.2% when removed | Improved |
| add working memory to LIGHT (ablation) | LIGHT · BEAM 100K-1M | performance: removal changes -1.6% at 100K, slightly improves at 500K and 1M | No clear effect |
| add retrieval to LIGHT (ablation) | LIGHT · BEAM 10M | performance: -8.5% when removed | Improved |
| add scratchpad to LIGHT (ablation) | LIGHT · BEAM 10M | performance: -3.7% when removed | Improved |
| add working memory to LIGHT (ablation) | LIGHT · BEAM 10M | performance: -5.7% when removed | Improved |
| add noise filtering to LIGHT (ablation) | LIGHT · BEAM 10M | performance: -8.3% when removed | Improved |
Closest papers by meaning
- LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory (ICLR 2025 · similarity 0.91)
- Human-inspired Episodic Memory for Infinite Context LLMs (ICLR 2025 · similarity 0.91)
- From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMs (ICLR 2025 · similarity 0.91)
- Episodic Memories Generation and Evaluation Benchmark for Large Language Models (ICLR 2025 · similarity 0.90)
- Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.90)
- Enhancing LLM’s Cognition via Structurization (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
