Research map / Video / Action recognition
Learning Procedure-Aware Video Representation From Instructional Videos and Their Narrations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add diffusion-based temporal order modeling to matching pre-training (ablation) | Video representation learning · COIN step classification, zero-shot | Top-1 accuracy (%): 13.7% to 16.6% | Helped |
| Add temporal order modeling to matching pre-training (ablation) | Video representation learning · COIN step forecasting, zero-shot | Top-1 accuracy (%): No zero-shot ability -> 11.3% | Helped |
| Add temporal order modeling to matching pre-training (ablation) | Video representation learning · COIN step forecasting, fine-tuning | Top-1 accuracy (%): +1.8% | Helped |
| Use diffusion instead of masked prediction for temporal order modeling (ablation) | Procedure-aware video representation · COIN step forecasting, fine-tuning | Top-1 accuracy (%): +3.6% | Helped |
| Use diffusion instead of masked prediction for temporal order modeling (ablation) | Procedure-aware video representation · COIN step forecasting, zero-shot | Top-1 accuracy (%): +0.7% | Helped |
| Replace zero-vector approximate inference with Monte Carlo estimation (ablation) | Ours · COIN step forecasting, zero-shot | Top-1 accuracy (%): 11.33 -> 11.34 ± 0.03 | No clear effect |
| Use oracle ranking to choose among multiple sampled predictions (ablation) | Ours · COIN step forecasting, zero-shot | Top-1 accuracy (%): 11.33 -> 14.73 ± 0.13 | Helped |
| Use ASR phrases instead of wikiHow sentences (ablation) | Pre-training step descriptions · COIN step forecasting, zero-shot | Top-1 accuracy (%): 8.3 -> 9.0 | Helped |
| Use ASR phrases instead of wikiHow sentences (ablation) | Pre-training step descriptions · COIN step forecasting, fine-tuning | Top-1 accuracy (%): 38.0 -> 38.9 | Helped |
| Replace TimeSformer with MViT-S (ablation) | Video encoder · COIN step classification, zero-shot | Top-1 accuracy (%): 16.6 -> 12.5 | Hurt |
| Replace TimeSformer with MViT-S (ablation) | Video encoder · COIN step forecasting, zero-shot | Top-1 accuracy (%): 11.3 -> 9.0 | Hurt |
| Use procedure-aware matching and temporal ordering rather than matching only (ablation) | Video representation learning · COIN step classification, fine-tuning | Top-1 accuracy (%): 52.8 -> 56.9 | Helped |
Closest papers by meaning
- Procedure Knowledge Decoupled Distillation Strategy for Procedure Planning in Instructional Videos (AAAI 2025 · similarity 0.90)
- Why Not Use Your Textbook? Knowledge-Enhanced Procedure Planning of Instructional Videos (CVPR 2024 · similarity 0.89)
- SCHEMA: State CHangEs MAtter for Procedure Planning in Instructional Videos (ICLR 2024 · similarity 0.89)
- Masked Temporal Interpolation Diffusion for Procedure Planning in Instructional Videos (ICLR 2025 · similarity 0.88)
- Collaborative Weakly Supervised Video Correlation Learning for Procedure-Aware Instructional Video Analysis (AAAI 2024 · similarity 0.88)
- ShowHowTo: Generating Scene-Conditioned Step-by-Step Visual Instructions (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.