Research map / Reinforcement learning / Visual and representation RL
Visual Forecasting by Imitating Dynamics in Natural Sequences
Kuo-Hao Zeng, William B. Shen, De-An Huang, Min Sun, Juan Carlos Niebles
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add IRL long-term cost (sequence modeling) instead of local GAN cost (ablation) | IRL frame generator · Moving MNIST (human evaluation) | accuracy at fooling annotators (%): 10.2 -> 15.1 | Improved |
| replace L1 reconstruction loss with IRL | DCGAN-based frame generator · Moving MNIST (human evaluation) | accuracy at fooling annotators (%): 0.3 -> 15.1 | Improved |
| replace LSTM predictor with IRL | frame predictor · Moving MNIST (human evaluation) | accuracy at fooling annotators (%): 1.4 -> 15.1 | Improved |
| replace L2 regression loss with IRL (ablation) | Deep Regression (K=1) · TV Human Interactions | accuracy (%): 37.5 -> 45.8 | Improved |
| replace LSTM reconstruction-loss sequence model with IRL | LSTM · TV Human Interactions | accuracy (%): 40.5 -> 45.8 | Improved |
| add IRL single-mode policy instead of explicit mixture regression | Deep Regression (K=3) · TV Human Interactions | accuracy (%): 44.0 -> 45.8 | Improved |
| add deep representation reparametrization instead of pixel synthesis (ablation) | Pixel Synthesis · TV Human Interactions | accuracy (%): 26.1 -> 45.8 | Improved |
| train SVM one second before action instead of during action | SVM · TV Human Interactions | accuracy (%): 36.2 -> 35.8 | No clear effect |
| add IRL instead of nearest neighbor for long-term storyline prediction | storyline forecasting · Visual Storytelling long-term | accuracy | Improved |
| use nearest neighbor ResNet-101 features instead of IRL for short-term storyline prediction | storyline forecasting · Visual Storytelling short-term | accuracy | Hurt |
Closest papers by meaning
- First-Person Activity Forecasting With Online Inverse Reinforcement Learning (ICCV 2017 · similarity 0.88)
- Robust Visual Imitation Learning with Inverse Dynamics Representations (AAAI 2024 · similarity 0.87)
- VEDIT: Latent Prediction Architecture For Procedural Video Representation Learning (ICLR 2025 · similarity 0.87)
- Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting (ICML 2025 · similarity 0.86)
- PlaySlot: Learning Inverse Latent Dynamics for Controllable Object-Centric Video Prediction and Planning (ICML 2025 · similarity 0.85)
- VisionTS: Visual Masked Autoencoders Are Free-Lunch Zero-Shot Time Series Forecasters (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
