Research map / Video / Video segmentation and tracking
Visual Dynamics: Stochastic Future Generation via Layered Cross Convolutional Networks
Tianfan Xue, Jiajun Wu, Katherine L. Bouman, William T. Freeman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the proposed model instead of Reed et al. Dis+Cls for analogy-making | Cross Convolutional Network · Sprites visual analogy | average mean squared pixel error: 23.0 -> 15.9 | Improved |
| use the proposed model instead of Reed et al. Add | Cross Convolutional Network · Sprites visual analogy | average mean squared pixel error: 56.0 -> 15.9 | Improved |
| use the proposed model instead of Dis+Cls on slash animation | Cross Convolutional Network · Sprites visual analogy (slash) | mean squared pixel error: 18.9 -> 28.2 | Hurt |
| replace the VAE with a standard autoencoder (remove KL loss and sampling) (ablation) | Cross Convolutional Network · Shapes | KL divergence to ground-truth motion distribution | Improved |
| use cross-convolutional VAE instead of nearest-neighbor flow copying | Cross Convolutional Network · Shapes | KL divergence to ground-truth motion distribution | Improved |
| use cross-convolutional VAE instead of warping by closest training flow | Cross Convolutional Network · Sprites | % labeled real by humans | Improved |
| increase resolution from 32x32 to 64x64 | Cross Convolutional Network · Sprites | % labeled real by humans: 41.2 -> 35.7 | Hurt |
| increase resolution from 32x32 to 64x64 | Cross Convolutional Network · Exercise | % labeled real by humans: 36.7 -> 31.3 | Hurt |
| shrink latent z from 3200 to 8, 32 or 128 dimensions (ablation) | Cross Convolutional Network · Shapes | convergence and quality | No clear effect |
| increase KL weight lambda (ablation) | Cross Convolutional Network · Shapes | sparsity of z | Improved |
| use more complex motion data (Exercise vs Sprites) | Cross Convolutional Network · latent z | non-zero elements in z_mean: 54 -> 978 | Mixed |
Closest papers by meaning
- Visual Dynamics: Probabilistic Future Frame Synthesis via Cross Convolutional Networks (NeurIPS 2016 · similarity 0.97)
- Visual Interaction Networks: Learning a Physics Simulator from Video (NeurIPS 2017 · similarity 0.89)
- Probabilistic Future Prediction for Video Scene Understanding (ECCV 2020 · similarity 0.88)
- Visual Representation Learning with Stochastic Frame Prediction (ICML 2024 · similarity 0.87)
- E-Motion: Future Motion Simulation via Event Sequence Diffusion (NeurIPS 2024 · similarity 0.87)
- Generative Image Dynamics (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
