Research map / Video / Video segmentation and tracking
The Pose Knows: Video Forecasting by Generating Pose Futures
Jacob Walker, Kenneth Marino, Abhinav Gupta, Martial Hebert
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use pose-VAE plus pose-conditioned GAN instead of direct pixel Video-GAN | video forecasting on UCF101 · UCF101 | Inception score: 1.74 -> 3.14 | Improved |
| use pose-VAE plus pose-conditioned GAN instead of direct pixel Video-GAN | video forecasting on UCF101 · UCF101 | MMD: 0.139 -> 0.022 | Improved |
| add the VAE to the future decoder (versus deterministic ERD and SRNN) (ablation) | Pose-VAE · UCF101 pose forecasting | minimum Euclidean distance over n samples | Improved |
| make deterministic baselines stochastic using Gaussian noise with variance from test data (ablation) | ERD and SRNN · UCF101 pose forecasting | minimum Euclidean distance over n samples | Hurt |
| use recurrent Pose-VAE instead of feed-forward VAE (ablation) | Pose-VAE · UCF101 pose forecasting | minimum Euclidean distance over n samples | Improved |
| increase number of samples drawn from Pose-VAE | Pose-VAE · UCF101 pose forecasting | minimum Euclidean distance | Improved |
Closest papers by meaning
- Learning to Forecast and Refine Residual Motion for Image-to-Video Generation (ECCV 2018 · similarity 0.89)
- PoseGPT: Quantization-Based 3D Human Motion Generation and Forecasting (ECCV 2022 · similarity 0.89)
- FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observations (CVPR 2024 · similarity 0.89)
- Pose Guided Human Video Generation (ECCV 2018 · similarity 0.88)
- Video Prediction with Appearance and Motion Conditions (ICML 2018 · similarity 0.88)
- Compositional Video Prediction (ICCV 2019 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
