Research map / Architectures and efficiency / RNNs and normalization
Recurrent Independent Mechanisms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RIMs instead of LSTM | Copying task · Copying, test length 200 after training length 50 | CE: 3.56 -> 0.00 | Helped |
| Use RIMs instead of the strongest reported baseline | Sequential MNIST · 19 x 19 resolution | Accuracy: 54.23 -> 73.4 | Helped |
| Use RIMs instead of the strongest reported baseline | Sequential MNIST · 24 x 24 resolution | Accuracy: 27.75 -> 38.1 | Helped |
| Use RIMs instead of the strongest reported baseline | Sequential MNIST · 16 x 16 resolution | Accuracy: 91.2 -> 90.0 | Hurt |
| Remove communication between RIMs (ablation) | RIMs · Copying, test length 200 | CE: 0.00 -> 1.6 | Hurt |
| Remove input attention and activate all RIMs (ablation) | RIMs · Copying, test length 200 | CE: 0.00 -> 0.7 | Hurt |
| Use 9 RIMs and activate 3 (ablation) | RIMs · Copying, test length 200 | CE: 0.00 | No clear effect |
| Activate all 9 RIMs rather than a sparse subset (ablation) | RIMs · Copying, test length 200 | CE: 0.00 -> 0.24 | Hurt |
| Activate most RIMs (14 of 16) (ablation) | RIMs · Copying, test length 200 | CE: 0.00 -> 0.26 | Hurt |
| Activate most RIMs (22 of 24) (ablation) | RIMs · Copying, test length 200 | CE: 0.00 -> 0.42 | Hurt |
| Replace the LSTM with RIMs | Atari PPO agent · Atari games | relative score | Helped |
| Increase LSTM capacity from 256 to 512 units (ablation) | Bouncing Balls prediction · Novel object-count generalization | performance gap versus RIMs: 256 to 512 | No clear effect |
Closest papers by meaning
- Emergent mechanisms for long timescales depend on training curriculum and affect performance in memory tasks (ICLR 2024 · similarity 0.86)
- Flexible task abstractions emerge in linear networks with fast and bounded units (NeurIPS 2024 · similarity 0.85)
- Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks (NeurIPS 2025 · similarity 0.85)
- Range, not Independence, Drives Modularity in Biologically Inspired Representations (ICLR 2025 · similarity 0.85)
- Learning the Plasticity: Plasticity-Driven Learning Framework in Spiking Neural Networks (NeurIPS 2025 · similarity 0.84)
- Slow and Steady Wins the Race: Maintaining Plasticity with Hare and Tortoise Networks (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.