Research map / Probabilistic and scientific modeling / Dynamical systems
Neural Foundations of Mental Simulation: Future Prediction of Latent Representations on Dynamic Scenes
Aran Nayebi, Rishi Rajalingham, Mehrdad Jazayeri, Guangyu Robert Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Future-predict in video foundation model latents using LSTM or CTRNN dynamics | VC-1 and R3M latent models · Mental-Pong | neural predictivity: 19.03% and 24.79% -> 46.34-48.83% | Improved |
| Use dynamic video-pretrained latents rather than static-image-pretrained latents | Latent future prediction models · Mental-Pong | neural predictivity: at most 29.67% -> 46.34-48.83% | Improved |
| Use VC-1 or R3M representations instead of VIP representations | Video foundation model latent predictors · Mental-Pong | neural predictivity: 7.37%-18.12% -> 46.34-48.83% | Improved |
| Add a dynamics module to the VC-1 or R3M encoder latents (ablation) | VC-1 and R3M · Mental-Pong | neural predictivity: 19.03% and 24.79% -> 46.34-48.83% | Improved |
| Increase SVG input resolution from 64×64 to 128×128 (ablation) | SVG · Physion OCP held-out scenes | accuracy | No clear effect |
| Add temporal RandAugment (ablation) | FitVid · Physion OCP held-out scenes | accuracy | No clear effect |
| Use CTRNN dynamics instead of LSTM dynamics (ablation) | VC-1 and R3M latent models · Mental-Pong | neural predictivity | Improved |
| Pretrain dynamics on Kinetics-700 instead of Physion (ablation) | VIP, VC-1, and R3M latent models · Physion OCP | accuracy | No clear effect |
| Pretrain dynamics on Kinetics-700 instead of Physion (ablation) | VIP, VC-1, and R3M latent models · Mental-Pong | neural predictivity | No clear effect |
| Use object-slot C-SWM instead of pixel-wise or latent prediction | C-SWM · Physion OCP | accuracy and human judgment probability match | Hurt |
| Use end-to-end pixel-wise prediction instead of latent video-model prediction | FitVid and SVG · Mental-Pong | neural predictivity: 14.44%-25.35% vs. 46.34-48.83% | Hurt |
| Use end-to-end pixel-wise prediction rather than latent video-model prediction | FitVid and SVG · Physion OCP and Mental-Pong | generalization across environments: 61.12%-63.31%; 14.44%-25.35% | Mixed |
Closest papers by meaning
- OmniMouse: Scaling properties of multi-modal, multi-task Brain Models on 150B Neural Tokens (ICLR 2026 · similarity 0.89)
- Flexible Context-Driven Sensory Processing in Dynamical Vision Models (NeurIPS 2024 · similarity 0.89)
- CogReact: A Reinforced Framework to Model Human Cognitive Reaction Modulated by Dynamic Intervention (ICML 2025 · similarity 0.88)
- Modeling Dynamic Neural Activity by combining Naturalistic Video Stimuli and Stimulus-independent Latent Factors (NeurIPS 2025 · similarity 0.88)
- CogDPM: Diffusion Probabilistic Models via Cognitive Predictive Coding (ICML 2024 · similarity 0.88)
- Learning 3D Persistent Embodied World Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
