Research map / Large language models / Language, safety and interpretability
Temporal Context Reinstatement Drives Episodic-Like Order Memory in Long-Context Language Models
Mathis Pink, Vy A. Vo, Qinyuan Wu, Jianing Mu, Javier S. Turek, Uri Hasson, Kenneth A. Norman, Sebastian Michelmann, Alexander Huth, Mariya Toneva
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the time-reinstatement head at retrieval (vs removing it) (ablation) | Mistral-7B-v0.2 · SORT | accuracy: 0.60 -> 0.50 | Improved |
| add the time-reinstatement head at retrieval (vs removing it) (ablation) | Qwen2.5-7B · SORT | accuracy: 0.65 -> 0.60 | Improved |
| add head L8H11 at retrieval (vs removing it), 500-word context (ablation) | Llama3.1-8B-Instruct · Dorian Gray ordering | Kendall tau: 0.63 -> 0.54 | Improved |
| add head L8H11 at retrieval (vs removing it), 25k-word context (ablation) | Llama3.1-8B-Instruct · Dorian Gray ordering | Kendall tau: 0.31 -> 0.11 | Improved |
| add temporal direction (vs projecting it out) in time-reinstatement head (ablation) | Llama3.1-8B and 70B · TMORA SORT | accuracy: 10-15% | Improved |
| amplify gain on L8H11 by factor 1.75 | Llama3.1-8B-Instruct · TMORA SORT | accuracy: up to 9% | Improved |
| shuffle text in blocks of four sentences (ablation) | Llama3.1-8B and 70B · TMORA shuffled | accuracy: p >= 0.307 (8B), p >= 0.104 (70B) | No clear effect |
| disable RoPE (ablation) | Llama3.1-8B-Instruct · random tokens, temporal info in value cache | Kendall tau: cosine similarity 0.69 | Improved |
| scale from 8B to 70B model vs humans | Llama3.1-70B-Instruct · TMORA 16k-24k words apart | accuracy: p = 0.021 | Improved |
| compare Llama3.1-8B with humans | Llama3.1-8B-Instruct · TMORA | accuracy: all p >= 0.289 | No clear effect |
Closest papers by meaning
- Human-inspired Episodic Memory for Infinite Context LLMs (ICLR 2025 · similarity 0.92)
- Episodic Memories Generation and Evaluation Benchmark for Large Language Models (ICLR 2025 · similarity 0.91)
- REMem: Reasoning with Episodic Memory in Language Agent (ICLR 2026 · similarity 0.89)
- Linking In-context Learning in Transformers to Human Episodic Memory (NeurIPS 2024 · similarity 0.88)
- Quantifying Memory Utilization with Effective State-Size (ICML 2025 · similarity 0.87)
- Scaling and context steer LLMs along the same computational path as the human brain (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
