Research map / Architectures and efficiency / RNNs and normalization
Geometry of naturalistic object representations in recurrent neural network models of working memory
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains CNN (ResNet50) + RNN (vanilla, GRU, LSTM) sensory-cognitive models on nine naturalistic-stimulus N-back tasks and analyzes RNN latent geometry (decoding, orthogonalization, Procrustes) to find chronological memory subspaces.
Problem
How naturalistic, multidimensional object information is encoded, maintained, and transformed in recurrent neural networks performing dynamic working-memory tasks (N-back), rather than categorical one-hot inputs or single fixation-delay tasks.
Approach
Couples a pretrained ResNet50 visual encoder with vanilla RNN/GRU/LSTM trained on nine N-back variants (N in 1-3 by feature location/identity/category) and uses decoder-based geometry and Procrustes rotation analyses to test slot-based versus relative chronological memory hypotheses.
Key results
- Gated RNNs (GRU, LSTM) use highly task-specific latent subspaces for object properties, whereas vanilla RNN subspaces are shared across tasks.
- RNN latent spaces slightly de-orthogonalize object feature axes relative to the perceptual (ResNet50) space, contrary to the hypothesis.
- Cross-time Procrustes rotations generalize across stimuli but not across time steps, supporting relative chronological memory subspaces (H2) over slot-based memory.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on novel object angles rather than training angles | Sensory-cognitive RNN models · N-back validation, novel object angles | accuracy | Helped |
| Evaluate on novel object identities instead of novel view angles | Sensory-cognitive RNN models · N-back validation, novel object identities | accuracy | Hurt |
| Increase the number of model parameters | RNN models · N-back | accuracy | Helped |
| Use a vanilla RNN instead of a gated recurrent architecture at identical parameter count | Sensory-cognitive RNN models · N-back | accuracy | Hurt |
| Train on multiple features/tasks rather than a single task and feature (ablation) | GRU models · N-back, task-irrelevant feature decoding | decoding accuracy | Helped |
| Use multi-task training with a vanilla RNN rather than a gated RNN (ablation) | MTMF models · N-back, cross-task object-property decoding | generalization accuracy | Helped |
| Use gated RNNs rather than a vanilla RNN (ablation) | MTMF models · N-back, cross-task object-property decoding | generalization accuracy | Hurt |
| Transform perceptual representations into RNN encoding representations | Sensory-cognitive RNN models · Naturalistic object-property representations | orthogonalization index | Hurt |
| Use a temporally fixed memory subspace rather than chronological memory subspaces | RNN models · N-back, cross-time decoding | decoding accuracy | Hurt |
| Use stimulus-specific encoding subspaces rather than shared chronological encoding subspaces | RNN models · N-back, cross-stimulus encoding decoding | generalization accuracy | Hurt |
| Swap in a rotation matrix from another stimulus at the same chronological transition | MTMF RNN models · N-back, reconstructed-decoder accuracy | decoding accuracy | Helped |
| Swap in a rotation matrix from the same stimulus at an adjacent time transition | MTMF RNN models · N-back, reconstructed-decoder accuracy | decoding accuracy | Hurt |
Surprising findings
- Object features are less orthogonalized in RNN latent space than in the perceptual space, opposite to the hypothesis.
- Multi-task models retain task-irrelevant object properties as well as task-relevant ones.
Limitations
- Results are specific to the N-back task structure and may not transfer to other WM tasks.
- Generalization to novel object identities is substantially weaker and not analyzed geometrically.
- Only vanilla RNN, GRU, and LSTM architectures studied; network scaling not explored.
Experimental setup
Datasets: ShapeNet ImageNet iWISDM-generated N-back trials
Benchmarks: N-back tasks (1/2/3-back; location, identity, category)
Metrics: accuracy decoding accuracy orthogonalization index Procrustes reconstruction accuracy
Baselines: vanilla RNN GRU LSTM
Builds on: ResNet GRU LSTM ShapeNet iWISDM
Closest papers by meaning
- Evaluating alignment between humans and neural network representations in image-based learning tasks (NeurIPS 2024 · similarity 0.88)
- Flexible Context-Driven Sensory Processing in Dynamical Vision Models (NeurIPS 2024 · similarity 0.88)
- Task structure and nonlinearity jointly determine learned representational geometry (ICLR 2024 · similarity 0.87)
- RNNs perform task computations by dynamically warping neural representations (NeurIPS 2025 · similarity 0.87)
- Separating the 'what' and 'how' of compositional computation to enable reuse and continual learning (NeurIPS 2025 · similarity 0.86)
- Learning dynamics in linear recurrent neural networks (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.