Research map / Video / Video-language understanding
Gaussian Mixture Proposals with Pull-Push Learning Scheme to Capture Diverse Events for Weakly Supervised Temporal Video Grounding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Gaussian mixture proposals (GMP) with a pull-push learning scheme and importance-weighted reconstructor for weakly supervised temporal video grounding, achieving state-of-the-art recall on ActivityNet Captions and Charades-STA.
Problem
Weakly supervised temporal video grounding relies on single predetermined Gaussian proposals that cannot express diverse query-relevant events described by a sentence.
Approach
Represent each proposal as a mixture of learnable Gaussian masks (learned importance, centroid, width) over temporal locations, and train them with a pulling loss (centroids pulled together) and intra/inter pushing losses (masks and proposals kept less overlapped), so the Gaussians are moderately coupled; importance weights come from a mask-conditioned reconstructor.</idea>
Key results
- On ActivityNet Captions, PPS reaches R@1,IoU=0.3 of 59.29 and R@5,IoU=0.5 of 71.32, outperforming CPL by 3.56 and 28.19 points at those settings.
- On Charades-STA, PPS achieves R@1,IoU=0.7 of 26.16, surpassing CPL by 3.77 points.
- Ablation shows removing the pulling loss drops ActivityNet R@1,IoU=0.3 performance substantially, and the full three-loss combination performs best (59.29).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the single Gaussian with a Gaussian mixture proposal (ablation) | PPS · ActivityNet Captions | R@1, IoU=0.3: 47.49 -> 59.29 | Helped |
| Use importance weights from the reconstructor rather than no importance weighting (ablation) | PPS · ActivityNet Captions | R@1, IoU=0.3: 52.55 -> 59.29 | Helped |
| Estimate importance from the reconstructor rather than the generator (ablation) | PPS · ActivityNet Captions | R@1, IoU=0.3: 48.55 -> 59.29 | Helped |
| Learn multiple centers and one width per mixture rather than one center and multiple widths (ablation) | PPS · ActivityNet Captions | R@1, IoU=0.3: 46.63 -> 59.29 | Helped |
| Vary the number of masks across positive proposals instead of fixing it (ablation) | PPS · ActivityNet Captions | R@1 mIoU: 33.33 -> 37.59 | Helped |
| Add the pulling loss (ablation) | PPS pull-push scheme · ActivityNet Captions | R@1, IoU=0.3: 45.23 -> 55.03 | Helped |
| Use all three losses rather than the pulling loss alone (ablation) | PPS pull-push scheme · ActivityNet Captions | R@1, IoU=0.3: 55.03 -> 59.29 | Helped |
| Use all three losses rather than pulling and intra-pushing only (ablation) | PPS pull-push scheme · ActivityNet Captions | R@1, IoU=0.3: 49.98 -> 59.29 | Helped |
| Pull the two most distant masks together rather than pulling all masks (ablation) | PPS pulling loss · ActivityNet Captions | R@1 mIoU: 35.41 -> 37.59 | Helped |
| Use PPS rather than CPL | Temporal video grounding · ActivityNet Captions | R@5, IoU=0.5: 66.61 -> 71.32 | Helped |
| Use PPS rather than CPL | Temporal video grounding · Charades-STA | R@1, IoU=0.7: 22.39 -> 26.16 | Helped |
Surprising findings
- Learning multiple widths within a mixture performs worse than a single shared width, attributed to added complexity.
- Using only the pulling loss already yields results competitive with prior state-of-the-art methods.
Limitations
- Only Gaussian-mixture shapes are explored; other shapes for complex temporal structures are not investigated.
Experimental setup
Datasets: ActivityNet Captions Charades-STA
Benchmarks: ActivityNet Captions Charades-STA
Metrics: R@1,IoU=0.3 R@1,IoU=0.5 R@1,IoU=0.7 R@5,IoU=0.3 R@5,IoU=0.5 R@1,mIoU R@5,mIoU
Baselines: CPL IRON CRM CNM LCNet WSTAN VCA RTBPN
Builds on: CPL reconstruction-based weakly supervised grounding GloVe C3D
Closest papers by meaning
- Local-Global Multi-Modal Distillation for Weakly-Supervised Temporal Video Grounding (AAAI 2024 · similarity 0.83)
- Contextual Self-paced Learning for Weakly Supervised Spatio-Temporal Video Grounding (ICLR 2025 · similarity 0.83)
- STPro: Spatial and Temporal Progressive Learning for Weakly Supervised Spatio-Temporal Grounding (CVPR 2025 · similarity 0.83)
- DeCafNet: Delegate and Conquer for Efficient Temporal Grounding in Long Videos (CVPR 2025 · similarity 0.82)
- Uncertainty-quantified Rollout Policy Adaptation for Unlabelled Cross-domain Video Temporal Grounding (NeurIPS 2025 · similarity 0.82)
- SnAG: Scalable and Accurate Video Grounding (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.