Research map / Probabilistic and scientific modeling / Dynamical systems
ModeRNN: Harnessing Spatiotemporal Mode Collapse in Unsupervised Predictive Learning
Zhiyu Yao, Yunbo Wang, Haixu Wu, Jianmin Wang, Mingsheng Long
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on entire mixed dataset instead of single-mode subset | ConvLSTM · Mixed Moving MNIST Mode-1 | MSE: 26.6 -> 33.4 | Hurt |
| train on entire mixed dataset instead of single-mode subset | ConvLSTM · Mixed Moving MNIST Mode-3 | MSE: 120.5 -> 137.1 | Hurt |
| train on entire mixed dataset instead of single-mode subset | ModeRNN · Mixed Moving MNIST Mode-1 | MSE: 21.9 -> 17.1 | Improved |
| train on entire mixed dataset instead of single-mode subset | ModeRNN · Mixed Moving MNIST Mode-3 | MSE: 83.1 -> 74.8 | Improved |
| train on entire mixed dataset instead of subset | RIM · Mixed Moving MNIST Mode-2 | MSE: 53.3 -> 52.1 | Mixed |
| increase spatiotemporal slots from 1 to 4 (ablation) | ModeRNN · Mixed Moving MNIST | per-frame MSE: 69.3 -> 44.7 | Improved |
| increase slots from 4 to 5 (ablation) | ModeRNN · Mixed Moving MNIST | per-frame MSE: 44.7 -> 45.9 | Hurt |
| add adaptive slot fusion instead of equal-weight sum (ablation) | ModeRNN · Mixed Moving MNIST | per-frame MSE: 60.5 -> 44.7 | Improved |
| add slot binding (multiple slots instead of N=1) (ablation) | ModeRNN · Mixed Moving MNIST | per-frame MSE: 69.3 -> 44.7 | Improved |
| replace baseline predictors with ModeRNN | ModeRNN vs SVG · KTH action | PSNR: 27.73 -> 28.22 | Improved |
| replace TrajGRU with ModeRNN | ModeRNN · Radar echo | CSI30: 0.357 -> 0.428 | Improved |
| train on entire KTH dataset instead of subsets | PredRNN · KTH Mode-1 (torso movement) | PSNR: 27.45 -> 26.92 | Hurt |
Closest papers by meaning
- PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learning (TPAMI 2022 · similarity 0.89)
- A Large-Scale Study on Unsupervised Spatiotemporal Representation Learning (CVPR 2021 · similarity 0.88)
- Wavelet-Driven Spatiotemporal Predictive Learning: Bridging Frequency and Time Variations (AAAI 2024 · similarity 0.87)
- UniSTD: Towards Unified Spatio-Temporal Learning across Diverse Disciplines (CVPR 2025 · similarity 0.86)
- Parallelized Spatiotemporal Slot Binding for Videos (ICML 2024 · similarity 0.85)
- StateSpaceDiffuser: Bringing Long Context to Diffusion World Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
