Research map / Architectures and efficiency / RNNs and normalization
Associative memory networks
Builds Hopfield-style and memory-augmented networks for associative recall and external memory.
48 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in RNNs and normalization, keyword cluster “end · differentiable · translation”
State History Tried Checklist Preprints Who Export
Where it stands
Not enough benchmark data Fewer than four papers report the same benchmark, so there is no record to follow.
11 papers in 2023–24 and 8 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 1.7% to 0.9%.
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes (NeurIPS 2016 · 57 citations)
- 2017 · Convolutional Sequence to Sequence Learning (ICML 2017 · 2,604 citations)
- 2018 · SkipFlow: Incorporating Neural Coherence Features for End-to-End Automatic Text Scoring (AAAI 2018 · 135 citations)
- 2019 · Global-to-local Memory Pointer Networks for Task-Oriented Dialogue (ICLR 2019 · 34 citations)
- 2020 · Self-Attentive Associative Memory (ICML 2020 · 23 citations)
- 2021 · Hopfield Networks is All You Need (ICLR 2021 · 189 citations)
- 2022 · Universal Hopfield Networks: A General Framework for Single-Shot Associative Memory Models (ICML 2022 · 13 citations)
- 2023 · Compacting Binary Neural Networks by Sparse Kernel Selection (CVPR 2023 · 5 citations)
- 2024 · Neural Bookmarks: Information Retrieval with Deep Learning and EEG Data (AAAI 2024 · 2 citations)
- 2025 · Nonparametric Modern Hopfield Models (ICML 2025 · 1 citation)
- 2026 · Geometric Entropy and Retrieval Phase Transitions in Continuous Thermal Dense Associative Memory (ICML 2026 · 0 citations)
Earliest papers here
- Scaling Memory-Augmented Neural Networks with Sparse Reads and Writes (NeurIPS 2016 · 57 citations)
- Convolutional Sequence to Sequence Learning (ICML 2017 · 2,604 citations)
- Learning to Remember Rare Events (ICLR 2017 · 236 citations)
What has been tried
39 papers here report 361 measured changes (a component added, removed or swapped, compared with a baseline): helped 68%, no clear effect 10%, hurt 16%, mixed 6%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| reduce training documents from 78 to 40 and EEG recording duration | EMD test topics · top-1 accuracy | 0.750 [0.71, 0.78] -> 0.476 [0.44, 0.51] | Hurt | Neural Bookmarks: Information Retrieval with Deep Learning and EEG Data (AAAI 2024 · 2 citations) |
| Remove global memory sharing across layers | Sort-of-CLEVR binary and ternary · accuracy (%) | 87.61 -> 80.29; 62.45 -> 60.06 | Hurt | A Framework for Inference Inspired by Human Memory Mechanisms (ICLR 2024 · 1 citation) |
| Use SinCos trigonometric random features instead of Positive Random Features | Letter · energy and gradient approximation quality | Helped | Dense Associative Memory Through the Lens of Random Features (NeurIPS 2024 · 1 citation) | |
| Replace Epanechnikov kernel with TriWeight kernel | Kernel comparison · emergent memories | Hurt | Dense Associative Memory with Epanechnikov Energy (NeurIPS 2025 · 0 citations) | |
| add categorical embedding | Datasets I average · AUC (%) | 90.82 -> 91.86 | Helped | BiSHop: Bi-Directional Cellular Learning for Tabular Data with Generalized Sparse Modern Hopfield Model (ICML 2024 · 0 citations) |
What reviewers will expect
Datasets and baselines named in the 10 papers of this approach from 2024–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- CIFAR-10 · 3 of 10 papers
- MNIST · 3 of 10 papers
- Tiny ImageNet · 3 of 10 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 19 papers; from OpenAlex and the papers' affiliations).
- Northwestern University · 5 papers
- National Taiwan University · 3 papers
- Massachusetts Institute of Technology · 2 papers
- Georgia Institute of Technology · 2 papers
- IBM · 2 papers
Use it
Download BibTeX of 35 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Differentiable optimization layers (125 papers)
- Neural-symbolic reasoning (98 papers)
- Non-autoregressive translation (102 papers)
- Learned embeddings for retrieval (76 papers)
Elsewhere in the topic
- Efficient network compression and training (229 papers)
- Theory of depth and width (223 papers)
- Representation geometry and collapse (169 papers)
- Dataset distillation and data-efficient training (158 papers)
- Neural scaling laws (155 papers)
- Distributed LLM inference systems (147 papers)
- Layer-wise analysis of language models (145 papers)
- Dataset condensation and flow estimation (126 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).