Research map / Architectures and efficiency / Attention and transformers
Relational recurrent neural networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use memory-slot interactions through multi-head dot-product attention | Recurrent memory on Nth Farthest · Nth Farthest, 16-dimensional vectors | best batch accuracy: LSTM and DNC failing to surpass 30%; RMC achieving 91% | Helped |
| Use memory-slot interactions through multi-head dot-product attention | Recurrent memory on BoxWorld · BoxWorld viewport | levels solved: 73% -> 98% | Helped |
| Use the RMC instead of an LSTM | Mini Pacman agent · Mini Pacman viewport | points: 550 -> 677 | Helped |
| Use the RMC instead of an LSTM | Mini Pacman agent · Mini Pacman full observation | points: 598 -> 1159 | Helped |
| Use the RMC instead of the strongest listed baseline | Word-based language modeling · WikiText-103 test | perplexity: 33 -> 31.6 | Helped |
| Use the RMC instead of the strongest listed baseline | Word-based language modeling · Project Gutenberg test | perplexity: 45.5 -> 42.0 | Helped |
| Use the RMC instead of the strongest listed baseline | Word-based language modeling · GigaWord v5 test | perplexity: 43.7 -> 38.3 | Helped |
| Train the decoder without teacher forcing, using model predictions (ablation) | Program Evaluation models · Learning to Execute tasks | test accuracy | Helped |
| Use one memory slot rather than multiple slots (ablation) | RMC on Nth Farthest · Nth Farthest | task performance | Hurt |
| Use more attention heads with a single memory slot (ablation) | RMC on Nth Farthest · Nth Farthest | task performance | Helped |
| Reduce test unroll length to 100 time steps (ablation) | RMC language model · Language modeling, test unroll length | perplexity regression: 1 perplexity | Hurt |
Closest papers by meaning
- A Framework for Inference Inspired by Human Memory Mechanisms (ICLR 2024 · similarity 0.87)
- When No Paths Lead to Rome: Benchmarking Systematic Neural Relational Reasoning (NeurIPS 2025 · similarity 0.87)
- A mathematical theory of balancing relational generalization and memorization (NeurIPS 2026 · similarity 0.86)
- Titans: Learning to Memorize at Test Time (NeurIPS 2025 · similarity 0.86)
- When to Memorize and When to Stop: Gated Recurrent Memory for Long-Context Reasoning (ICML 2026 · similarity 0.85)
- Emergent mechanisms for long timescales depend on training curriculum and affect performance in memory tasks (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.