Research map / Video / Video-language understanding
Visual-Textual Capsule Routing for Text-Based Video Segmentation
Bruce McIntosh, Kevin Duarte, Yogesh S Rawat, Mubarak Shah
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use visual-textual capsule routing for sentence-conditioned segmentation | Our Network · A2D | mAP: 21.5 -> 30.3 | Improved |
| Use visual-textual capsule routing for sentence-conditioned segmentation | Our Network · J-HMDB | mAP: 23.3 -> 26.1 | Improved |
| Train using bounding-box annotations for all video frames rather than key-frame annotations | Our Network · A2D full-video bounding-box evaluation | Video mAP: 21.2 -> 23.3 | Improved |
| Add capsule masking and classification loss (ablation) | Our Network · A2D | Mean IoU: 43.6 -> 46.0 | Improved |
| Add capsule masking while retaining classification loss (ablation) | Our Network · A2D | Mean IoU: 42.5 -> 46.0 | Improved |
| Replace concatenation with visual-textual capsule routing (ablation) | Our Network · A2D | Mean IoU: 25.0 -> 46.0 | Improved |
| Replace multiplication with visual-textual capsule routing (ablation) | Our Network · A2D | Mean IoU: 35.0 -> 46.0 | Improved |
| Replace dynamic filtering of capsule poses with visual-textual capsule routing (ablation) | Our Network · A2D | Mean IoU: 42.7 -> 46.0 | Improved |
| Replace dynamic filtering of capsule activations with visual-textual capsule routing (ablation) | Our Network · A2D | Mean IoU: 43.0 -> 46.0 | Improved |
Closest papers by meaning
- Actor and Action Video Segmentation From a Sentence (CVPR 2018 · similarity 0.89)
- Asymmetric Cross-Guided Attention Network for Actor and Action Video Segmentation From Natural Language Query (ICCV 2019 · similarity 0.88)
- Context Modulated Dynamic Networks for Actor and Action Video Segmentation with Language Queries (AAAI 2020 · similarity 0.88)
- Boosting Weakly-Supervised Temporal Action Localization With Text Information (CVPR 2023 · similarity 0.87)
- Boundary Proposal Network for Two-stage Natural Language Video Localization (AAAI 2021 · similarity 0.87)
- Boundary-Aware Temporal Dynamic Pseudo-Supervision Pairs Generation for Zero-Shot Natural Language Video Localization (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
