Research map / Video / Video segmentation and tracking
Learning Semantic-Aware Dynamics for Video Prediction
Xinzhu Bei, Yanchao Yang, Stefano Soatto
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| model class-specific semantic dynamics and use semantic-informed dis-occlusion synthesis | video prediction · Cityscapes t+5 | MS-SSIM: 75.68 -> 83.51 | Improved |
| model class-specific semantic dynamics and use semantic-informed dis-occlusion synthesis | video prediction · Cityscapes t+5 | LPIPS: 16.50 -> 14.93 | Improved |
| model class-specific semantic dynamics and use semantic-informed dis-occlusion synthesis | video prediction · KITTI Raw | MS-SSIM: 79.28 -> 83.06; 67.65 -> 72.44; 60.77 -> 64.72 | Improved |
| model class-specific semantic dynamics and use semantic-informed dis-occlusion synthesis | video prediction · KITTI Raw t+3 | LPIPS: 24.61 -> 24.58 | Improved |
| model class-specific semantic dynamics and use semantic-informed dis-occlusion synthesis | video prediction · KITTI Raw t+5 | LPIPS: 30.49 -> 31.16 | Hurt |
| model class-specific semantic dynamics and use semantic-informed dis-occlusion synthesis | video prediction · KITTI Flow next-frame | PSNR: 22.3 -> 24.47 | Improved |
| model class-specific semantic dynamics and use semantic-informed dis-occlusion synthesis | video prediction · KITTI Flow next-frame | SSIM: 69.6 -> 71.1 | Improved |
| model class-specific semantic dynamics and use semantic-informed dis-occlusion synthesis | video prediction · KITTI Flow next-frame | LPIPS: 11.4 -> 10.9 | Improved |
| merge static classes to use 9 classes rather than 20 (ablation) | SADM · Cityscapes video prediction t+5 | MS-SSIM: 79.14 -> 83.51 | Improved |
| merge static classes to use 9 classes rather than 20 (ablation) | SADM · Cityscapes video prediction t+5 | LPIPS: 17.55 -> 14.93 | Improved |
| replace class-specific dynamics with a single-class model using concatenated masks and flow (ablation) | SADM · video prediction | motion accuracy near semantic boundaries | Hurt |
| swap car and road semantic encoders (ablation) | SADM · qualitative motion prediction | class-consistent motion | Hurt |
Closest papers by meaning
- Representations Before Pixels: Semantics-Guided Hierarchical Video Prediction (ECCV 2026 · similarity 0.89)
- Disentangling Propagation and Generation for Video Prediction (ICCV 2019 · similarity 0.88)
- Revisiting Hierarchical Approach for Persistent Long-Term Video Prediction (ICLR 2021 · similarity 0.87)
- Semantic Flow: Learning Semantic Fields of Dynamic Scenes from Monocular Videos (ICLR 2024 · similarity 0.87)
- Compositional Video Prediction (ICCV 2019 · similarity 0.87)
- Learning Long-term Motion Embeddings for Efficient Kinematics Generation (CVPR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
