Research map / Video / Action recognition
End-to-End Learning of Visual Representations From Uncurated Instructional Videos
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the single positive pair with 3–5 nearby narration candidates (MIL-NCE) (ablation) | joint video-text embedding training · YouCook2 R@10 | R@10: 29.1 -> 35.0 | Helped |
| Increase positive candidates from 5 to 33 (ablation) | MIL-NCE · YouCook2 R@10 | R@10: 35.0 -> 28.3 | Hurt |
| Use NCE instead of binary classification loss (ablation) | single-instance video-text embedding training · YouCook2 R@10 | R@10: 18.5 -> 29.1 | Helped |
| Use NCE instead of max margin loss (ablation) | single-instance video-text embedding training · MSR-VTT R@10 | R@10: 24.1 -> 27.0 | Helped |
| Increase negatives per positive from 64 to 512 (ablation) | NCE training · MSR-VTT R@10 | R@10: 25.5 -> 29.0 | Helped |
| Use MIL-NCE rather than concatenating candidate narrations (Cat+NCE) (ablation) | multiple-positive training · YouCook2 R@10 | R@10: 31.9 -> 35.0 | Helped |
| Use MIL-NCE rather than max-pooling positive candidates (Max+NCE) (ablation) | multiple-positive training · YouCook2 R@10 | R@10: 32.3 -> 35.0 | Helped |
| Use MIL-NCE rather than attention selection (Attn+NCE) (ablation) | multiple-positive training · YouCook2 R@10 | R@10: 32.4 -> 35.0 | Helped |
| Sample negatives jointly for video and narration instead of video-only negatives (ablation) | MIL-NCE · YouCook2 R@10 | R@10: 34.4 -> 35.0 | Helped |
| Sample negatives jointly for video and narration instead of narration-only negatives (ablation) | MIL-NCE · YouCook2 R@10 | R@10: 19.3 -> 35.0 | Helped |
| Replace the paper's text model with BERT word vectors (ablation) | video-text embedding model · YouCook2 R@10 | R@10: 35.0 -> 17.6 | Hurt |
| Replace the paper's text model with BERT word vectors (ablation) | video-text embedding model · MSR-VTT R@10 | R@10: 31.8 -> 23.9 | Hurt |
Closest papers by meaning
- Unsupervised Learning of Action Classes With Continuous Temporal Embedding (CVPR 2019 · similarity 0.87)
- Frame-Wise Action Representations for Long Videos via Sequence Contrastive Learning (CVPR 2022 · similarity 0.87)
- Learning by Aligning Videos in Time (CVPR 2021 · similarity 0.86)
- Semi-Weakly-Supervised Learning of Complex Actions From Instructional Task Videos (CVPR 2022 · similarity 0.86)
- Unsupervised Learning From Narrated Instruction Videos (CVPR 2016 · similarity 0.86)
- Align Before Adapt: Leveraging Entity-to-Region Alignments for Generalizable Video Action Recognition (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.