Research map / Video / Video segmentation and tracking
What Happens Next? Anticipating Future Motion by Generating Point Trajectories
Gabrijel Boduljak, Laurynas Karazija, Iro Laina, Christian Rupprecht, Andrea Vedaldi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| generate point trajectories instead of RGB pixels (same architecture) (ablation) | flow-matching generator · Kubric in-distribution | LRTL: 33.7 -> 15.9 | Improved |
| jointly generate RGB and trajectories (ablation) | SD3.5 video generator · Kubric in-distribution | LRTL: 33.7 -> 28.2 | Improved |
| add trajectory VAE latent space (ablation) | Ours (B) · Kubric | Best-of-K MSE: 137.8 -> 127.2 | Improved |
| add trajectory VAE latent space (ablation) | Ours (B) · Physics101 | MSE overall: 28.62 -> 31.67 | Hurt |
| add trajectory VAE latent space (ablation) | Ours (B) · LIBERO-10 Effector (Mean) | MSE: 92.98 -> 66.35 | Improved |
| increase grid density from 8x8 to 32x32 (ablation) | Ours (B) · Kubric | Best-Of-K MSE: 147.37 -> 78.29 | Improved |
| scale denoiser from base to large at 32x32 (ablation) | Ours · Kubric | Best-Of-K MSE: 78.29 -> 72.75 | Improved |
| train on CoTracker pseudo trajectories instead of ground truth (ablation) | Ours (L) · Kubric | Best of K: 127.0 -> 140.3 | Hurt |
| fine-tune video generators on Kubric (ablation) | WAN 1.3B · Kubric in-distribution | FVMD: 23608 -> 14864 | Improved |
| use two input frames instead of one (ablation) | multi-frame Ours · Kubric in-distribution | MSE (Mean): 478.369 -> 264.963 | Improved |
| use small denoiser instead of large (ablation) | Ours · Kubric | LRTL: 14.6 -> 23.6 | Hurt |
| generate multiple samples and regress mean rather than ATM single regression | trajectory generation vs ATM · LIBERO-90 Effector | MSE: 67.37 -> 52.70 | Improved |
Closest papers by meaning
- Envisioning the Future, One Step at a Time (CVPR 2026 · similarity 0.91)
- Motion Modes: What Could Happen Next? (CVPR 2025 · similarity 0.89)
- How Far Is Video Generation from World Model: A Physical Law Perspective (ICML 2025 · similarity 0.88)
- MolmoMotion: Forecasting Point Trajectories in 3D with Language Instruction (NeurIPS 2026 · similarity 0.88)
- E-Motion: Future Motion Simulation via Event Sequence Diffusion (NeurIPS 2024 · similarity 0.88)
- Generative Image Dynamics (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
