Research map / Video / Action recognition
Learning Temporal Dynamics From Cycles in Narrated Video
Dave Epstein, Jiajun Wu, Cordelia Schmid, Chen Sun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a temporal constraint (max-index plus similarity loss) (ablation) | MMCC · Recipes test | cycle percentile rank: 92.6 -> 93.6 | Improved |
| add any temporal constraint against collapse (ablation) | MMCC · Recipes test | training stability: None -> 93.6 | Improved |
| sample start nodes by cross-modal concreteness instead of randomly (ablation) | MMCC · Recipes test | cycle percentile rank: 88.7 -> 93.6 | Improved |
| fine-tune input embeddings instead of freezing cross-modal features (ablation) | MMCC · Recipes test | cycle percentile rank: 67.5 -> 93.6 | Improved |
| fine-tune input embeddings instead of freezing cross-modal features (ablation) | MMCC · Recipes test | cross-modal percentile rank: 76.8 -> 75.7 | Hurt |
| cycle within modalities instead of across modalities (ablation) | MMCC · Recipes test | cycle percentile rank: 85.0 -> 93.6 | Improved |
| mix unimodal and multimodal states (p_unimodal 0.5) instead of unimodal only (ablation) | MMCC · Recipes test | cycle percentile rank: 89.8 -> 93.6 | Improved |
| use p_unimodal 0 (always multimodal) instead of 0.5 (ablation) | MMCC · Recipes test | cycle percentile rank: 93.6 -> 96.5 | Improved |
| add the cross-modal loss during co-training (ablation) | MMCC · Recipes test | cycle accuracy: 7% decrease without it | Improved |
| add the visual similarity loss (ablation) | MMCC · Recipes test | cycle rank: -1% without it | Improved |
| use MMCC instead of TAP for future prediction | TAP · CrossTask future action anticipation | recall@10: 27.9 -> 33.8 | Improved |
| use MMCC instead of TAP for unshuffling | TAP · Unshuffling image collections | Kendall's tau: 0.3344 -> 0.3632 | Improved |
Closest papers by meaning
- Time Is MattEr: Temporal Self-supervision for Video Transformers (ICML 2022 · similarity 0.88)
- Temporal Cycle-Consistency Learning (CVPR 2019 · similarity 0.88)
- Learning Temporal Transformations from Time-Lapse Videos (ECCV 2016 · similarity 0.88)
- Seeing Fast and Slow: Learning the Flow of Time in Videos (ECCV 2026 · similarity 0.88)
- Unsupervised Learning of Long-Term Motion Dynamics for Videos (CVPR 2017 · similarity 0.88)
- Towards Neuro-Symbolic Video Understanding (ECCV 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
