Research map / Graph learning / Knowledge and temporal graphs
Semantically-correlated memories in a dense associative model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Correlated Dense Associative Memory (CDAM), a continuous-valued dense associative memory whose graph-structured hetero-association is mixed with auto-association, analysed theoretically and in simulations over four dynamical modes.
Problem
Dense associative memories are largely auto-associative, and hetero-association over semantically correlated, continuous-valued memories with a controllable mix of auto- and hetero-association is under-studied; the notion of capacity is also unclear for such mixtures.
Approach
Retrieval projection Q = aΞ + hΞMᵀ, where M is the normalised adjacency matrix of an arbitrary memory graph, combined with softmax-based dense associative dynamics; varying the signs of a and h (including anti-Hebbian auto-association) yields auto-association, narrow/wide hetero-association and neutral quiescence, and controls the range of recalled correlated memories.
Key results
- Dynamics on cycle graph C30 show a controllable recall range from 0 to about 6 graph hops (one-way ANOVA F=5.41, p=0.001).
- Choices of a and h achieve R^2=0.997 fit with the monkey temporal cortex experimental data.
- On FashionMNIST auto-association, a=h=0.5-style mixtures outperform pure auto-association (a=1,h=0), which shows catastrophic forgetting as sudden performance drops.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| vary auto- and hetero-association strengths a and h | CDAM · Cycle graph, C_30 | correlation range (graph distance): 0 to ∼ 6 | Helped |
| vary auto- and hetero-association strengths across tested conditions | CDAM · Cycle graph, C_30 | mean correlation within 10-hop neighbourhoods: F=5.41, p-value =0.001 | Mixed |
| use anti-Hebbian auto-association (a<0) with relatively strong Hebbian hetero-association (h>0) | CDAM · Sparse real-video sequence recall | sequence recall (skips or delays) | Helped |
| set a=-2 and h=3 instead of other tested settings | CDAM · Sparse real-video sequence recall | sequence recall (skips or delays) | Helped |
| use a close to 0 and h close to 1, with a+h=1 | CDAM · FashionMNIST | highest-overlap pattern retrieval | Helped |
| set a=h (ablation) | CDAM · FashionMNIST | highest-overlap pattern retrieval | Hurt |
| remove hetero-association (set a=1, h=0) (ablation) | CDAM · FashionMNIST | highest-overlap pattern retrieval | Hurt |
| use a=1, h=0 instead of anti-Hebbian auto-association with Hebbian hetero-association (ablation) | CDAM · Sparse real-video sequence recall | sequence recall (skips or delays) | Hurt |
| use wide rather than narrow hetero-association (a=-0.5, h=1.5 rather than a=0.5, h=0.5) (ablation) | CDAM · 2-regular graph, random memory patterns | number and size of co-active metastable groups: a=-0.5,h=1.5; a=0.5,h=0.5 | Mixed |
| decrease a while keeping hetero-association active (ablation) | CDAM · Random 3-regular graph, p=46 | hetero-association spread | Helped |
| use anti-Hebbian auto-association to modulate the relative contribution of Hebbian hetero-association | CDAM · Memory graphs | community-structure representation scale | Helped |
| use anti-Hebbian modulation | CDAM · Temporal sequence recall | recall stability | Helped |
Surprising findings
- Pure auto-association (a=1, h=0) suffers catastrophic forgetting in the FashionMNIST task, while a=h performs very poorly.
- Pure pattern retrieval is impossible for non-isolated patterns when both a,h>0.
Limitations
- No formal definition of memory capacity for mixed auto- and hetero-association.
- Directed Erdos-Renyi transition result is only conjectured.
- Analysis of directed graphs is incomplete; numerical experiments are small-scale (n=1000).
Experimental setup
Datasets: FashionMNIST sparsely sampled video frames family tree image and text data
Benchmarks: FashionMNIST auto-association task Zachary's karate club Tutte graph barbell graph cycle graph C30 finite automaton family tree
Metrics: Pearson correlation between memory and state overlap-based prediction accuracy R^2 fit to experimental data
Baselines: classical Hopfield-style auto-associative model (Krotov-Hopfield Eq. 13) MESH
Builds on: modern Hopfield networks / dense associative memory Transformer attention as associative memory classical sequence-memory Hopfield models
Closest papers by meaning
- Factual recall in linear associative memories: sharp asymptotics and mechanistic insights (NeurIPS 2026 · similarity 0.89)
- Scaling Laws for Associative Memories (ICLR 2024 · similarity 0.89)
- Dense Associative Memory Through the Lens of Random Features (NeurIPS 2024 · similarity 0.88)
- Dense Associative Memory with Epanechnikov Energy (NeurIPS 2025 · similarity 0.87)
- Learning Associative Memories with Gradient Descent (ICML 2024 · similarity 0.87)
- On the Recall Scaling Laws in Mamba: A Theoretical and Mechanistic Study via Hashing (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.