Research map / Video / Video-language understanding
Modeling Video As Stochastic Processes for Fine-Grained Video Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Model frame sequences as Brownian-bridge processes and use process-based contrastive learning | Video representation learning · PennAction, Pouring, and IKEA ASM | Phase Classification: 93.07 -> 93.12; 93.73 -> 93.85; 26.80 -> 44.29 | Helped |
| Use phase-node boundaries to construct Brownian bridges | VSP · PennAction, Pouring, IKEA ASM, FineGym99, and FineGym288 | Phase Classification: 93.12 -> 93.27; 93.85 -> 94.08; 44.29 -> 46.77; 43.12 -> 44.58; 36.95 -> 38.23 | Helped |
| Use frame labels and supervised contrastive loss along with the process loss | VSP · PennAction, Pouring, IKEA ASM, FineGym99, and FineGym288 | Phase Classification: 93.27 -> 94.24; 94.08 -> 94.89; 46.77 -> 47.52; 44.58 -> 45.66; 38.23 -> 39.48 | Helped |
| Set bridge length η to 60 rather than the default 120 (ablation) | VSP · PennAction | Phase Classification: 93.12 -> 90.24 | Hurt |
| Set bridge length η to 240 rather than the default 120 (ablation) | VSP · PennAction | Phase Classification: 93.12 -> 91.38 | Hurt |
| Vary bridge length across batches using {60, 120, 240} (ablation) | VSP · PennAction | Phase Classification: 93.12 -> 88.70 | Hurt |
| Choose a random bridge length between [1, 240] (ablation) | VSP · PennAction | Phase Classification: 93.12 -> 74.83 | Hurt |
| Set adjacent-bridge overlap to 20% rather than 0%, 10%, or 50% (ablation) | VSP · PennAction | Phase Classification: 91.08 -> 93.12; 91.73 -> 93.12; 91.82 -> 93.12 | Helped |
| Set the LS:LP weight ratio to 1:0.1 rather than 1:1 (ablation) | VSP-F · PennAction | Phase Classification: 94.24 -> 94.45 | No clear effect |
| Set the LS:LP weight ratio to 1:10 rather than 1:1 (ablation) | VSP-F · PennAction | Phase Classification: 94.24 -> 91.32 | Hurt |
| Add intersecting Brownian bridges to provide hard positives (ablation) | VSP-F · PennAction | Phase Progression: 0.952 -> 0.814 | Hurt |
| Use supervised contrastive loss LS without process loss LP (ablation) | VSP-F · PennAction | Phase Progression: 0.952 -> 0.427 | Hurt |
Closest papers by meaning
- Probabilistic Representations for Video Contrastive Learning (CVPR 2022 · similarity 0.90)
- Frame-Wise Action Representations for Long Videos via Sequence Contrastive Learning (CVPR 2022 · similarity 0.88)
- Visual Representation Learning with Stochastic Frame Prediction (ICML 2024 · similarity 0.88)
- Towards Principled Representation Learning from Videos for Reinforcement Learning (ICLR 2024 · similarity 0.86)
- STDiff: Spatio-Temporal Diffusion for Continuous Stochastic Video Prediction (AAAI 2024 · similarity 0.85)
- Motion Consistency Model: Accelerating Video Diffusion with Disentangled Motion-Appearance Distillation (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.