Research map / Video / Action recognition
StepFormer: Self-Supervised Step Discovery and Localization in Instructional Videos
Nikita Dvornik, Isma Hadji, Ran Zhang, Konstantinos G. Derpanis, Richard P. Wildes, Allan D. Jepson
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train on subtitles and transfer without dataset-specific fine-tuning | StepFormer · CrossTask, ProceL, and COIN | F1 | Improved |
| Add sequence-to-sequence alignment loss (ablation) | StepFormer · CrossTask unsupervised segmentation | F1: 17.3 -> 28.3 | Improved |
| Add sequence-to-sequence alignment loss (ablation) | StepFormer · CrossTask zero-shot localization | IoU: 4.6 -> 23.7 | Improved |
| Add global contrastive loss (ablation) | StepFormer · CrossTask unsupervised segmentation | F1: 20.9 -> 28.3 | Improved |
| Add global contrastive loss (ablation) | StepFormer · CrossTask zero-shot localization | IoU: 11.5 -> 23.7 | Improved |
| Add diversity regularizer (ablation) | StepFormer · CrossTask unsupervised segmentation | F1: 26.7 -> 28.3 | Improved |
| Add diversity regularizer (ablation) | StepFormer · CrossTask zero-shot localization | IoU: 16.9 -> 23.7 | Improved |
| Add smoothness regularizer (ablation) | StepFormer · CrossTask unsupervised segmentation | F1: 27.1 -> 28.3 | Improved |
| Add smoothness regularizer (ablation) | StepFormer · CrossTask zero-shot localization | IoU: 17.7 -> 23.7 | Improved |
| Use ordered Drop-DTW inference instead of order-agnostic frame-to-slot assignment (ablation) | StepFormer · CrossTask unsupervised segmentation | F1: 28.1 -> 28.3 | No clear effect |
| Use ordered Drop-DTW inference instead of order-agnostic frame-to-slot assignment (ablation) | StepFormer · CrossTask zero-shot localization | IoU: 20.1 -> 23.7 | Improved |
| Use 48 rather than 32 step slots (ablation) | StepFormer · CrossTask unsupervised segmentation | F1: 28.3 -> 26.5 | Hurt |
Closest papers by meaning
- Flow Graph to Video Grounding for Weakly-Supervised Multi-step Localization (ECCV 2022 · similarity 0.91)
- Learning to Ground Instructional Articles in Videos through Narrations (ICCV 2023 · similarity 0.91)
- ShowHowTo: Generating Scene-Conditioned Step-by-Step Visual Instructions (CVPR 2025 · similarity 0.88)
- HR-Pro: Point-Supervised Temporal Action Localization via Hierarchical Reliability Propagation (AAAI 2024 · similarity 0.87)
- Collaborative Weakly Supervised Video Correlation Learning for Procedure-Aware Instructional Video Analysis (AAAI 2024 · similarity 0.86)
- Masked Temporal Interpolation Diffusion for Procedure Planning in Instructional Videos (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
