Research map / Architectures and efficiency / RNNs and normalization
Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a k-d tree approximate nearest neighbor index instead of exact linear search (ablation) | SAM · speed benchmark, 1M words | speed vs NTM: 100 -> 1600 | Helped |
| sparse reads and writes instead of dense memory | SAM · memory benchmark, 64K words | physical memory per 100 steps: 29 -> 7.8 | Helped |
| sparse variant (SAM) versus dense variant (DAM) (ablation) | SAM · copy, associative recall, priority sort | learning speed | Helped |
| sparse versus dense on the copy task (ablation) | SAM · Copy | training efficiency | No clear effect |
| scale memory to 2x10^6 words with sparse model under exponential curriculum | SAM · associative recall curriculum | max sequence length reached | Helped |
| train on length 10,000 then test on longer sequences | SAM · associative recall generalization | generalization length | Helped |
| use SDNC (sparse DNC) instead of dense DNC | SDNC · bAbI joint, best run | mean error (%): 5.2 -> 2.9 | Helped |
| use SAM instead of dense DAM (ablation) | SAM · bAbI joint, best run | mean error (%): 3.3 -> 4.1 | No clear effect |
| SDNC versus DNC averaged over 15 runs | SDNC · bAbI mean over 15 runs | mean error (%): 12.8 -> 6.4 | Helped |
| SDNC versus DNC at N=2048 | SDNC · speed and memory benchmark | speedup | Helped |
| larger K_L number of temporal links than 8 (ablation) | SDNC · learning | learning benefit | No clear effect |
| sparse model SAM on Omniglot compared to dense MANNs | SAM · Omniglot, 100 characters | errors: 0.4 -> 0.2 | Helped |
Closest papers by meaning
- Sparse Delta Memory: Scaling the State of Linear RNNs through Sparsity (NeurIPS 2026 · similarity 0.88)
- Hardware-aligned Hierarchical Sparse Attention for Efficient Long-term Memory Access (NeurIPS 2025 · similarity 0.87)
- Memory Layers at Scale (ICML 2025 · similarity 0.87)
- Blending Complementary Memory Systems in Hybrid Quadratic-Linear Transformers (NeurIPS 2025 · similarity 0.87)
- Language Model Memory and Memory Models for Language (NeurIPS 2026 · similarity 0.87)
- Memorization Sinks: Isolating Memorization during LLM Training (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.