Research map / Label-efficient and robust learning / Recognition and long-tail
Bridge the Gap: From Weak to Full Supervision for Temporal Action Localization with PseudoFormer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PseudoFormer, a two-branch weakly-supervised temporal action localization framework that fuses MIL proposals via Ricker-wavelet RickerFusion into better pseudo labels and trains an ActionFormer-style regression model with an uncertainty mask and iterative refinement.
Problem
Weakly-supervised temporal action localization (WTAL) lags fully-supervised TAL in both performance and framework; pseudo labels from weak models are noisy, priors are underused, and training with noisy pseudo labels is unresolved.
Approach
RickerFusion maps all predicted proposals into a shared global space using Ricker wavelets weighted by OIC confidence to produce cleaner pseudo proposals; a regression-based full branch learns from both proposal-level pseudo labels and snippet-level predictions, with an uncertainty mask that initially excludes proposal boundaries and is gradually relaxed, plus iterative proposal refinement.
Key results
- On THUMOS14, PseudoFormer with DELU reaches 51.9% average mAP at IoU 0.1:0.7, up 5.5 over the DELU base model (46.4%).
- On ActivityNet1.3, PseudoFormer achieves 29.0% average mAP (0.5:0.95), outperforming PivoTAL (28.1%).
- Integrated into DDG-Net, PseudoFormer reaches 52.4% average mAP on THUMOS14, surpassing PivoTAL by 2.8%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply PseudoFormer to DELU | DELU · THUMOS14 | mAP@IoU (0.1:0.7): 46.4 -> 51.9 | Helped |
| Apply PseudoFormer | Weakly-supervised temporal action localization · ActivityNet1.3 | mAP@IoU (0.5:0.95): 28.1 -> 29.0 | Helped |
| Add the regression-based full branch (ablation) | DELU base model · THUMOS14 | mAP@IoU (0.1:0.7): 46.4 -> 48.2 | Helped |
| Add RickerFusion (ablation) | Regression-based full branch with DELU · THUMOS14 | mAP@IoU (0.1:0.7): 48.2 -> 50.8 | Helped |
| Add attention loss (ablation) | Regression-based full branch with RickerFusion · THUMOS14 | mAP@IoU (0.1:0.7): 50.8 -> 51.3 | Helped |
| Add the uncertainty mask (ablation) | PseudoFormer with attention loss · THUMOS14 | mAP@IoU (0.1:0.7): 51.3 -> 51.6 | Helped |
| Add label refinement (ablation) | PseudoFormer with uncertainty mask · THUMOS14 | mAP@IoU (0.1:0.7): 51.6 -> 51.9 | Helped |
| Use hard proposal assignment instead of RickerFusion (ablation) | PseudoFormer pseudo-label generation · THUMOS14 | mAP@IoU (0.1:0.7): 46.0 vs 50.8 | Hurt |
| Use soft proposal assignment instead of RickerFusion (ablation) | PseudoFormer pseudo-label generation · THUMOS14 | mAP@IoU (0.1:0.7): 48.2 vs 50.8 | Hurt |
| Select top-K proposals instead of using RickerFusion (ablation) | PseudoFormer pseudo-label generation · THUMOS14 | mAP@IoU (0.1:0.7): 47.3 vs 50.8 | Hurt |
| Filter proposals by confidence threshold instead of using RickerFusion (ablation) | PseudoFormer pseudo-label generation · THUMOS14 | mAP@IoU (0.1:0.7): 49.0 vs 50.8 | Hurt |
| Use Gaussian proposal fusion instead of RickerFusion (ablation) | PseudoFormer pseudo-label generation · THUMOS14 | mAP@IoU (0.1:0.7): 49.2 vs 50.8 | Hurt |
Surprising findings
- Simple hard or soft proposal fusion of base-model outputs gave limited gains, while RickerFusion added about 2.6 points on THUMOS14 average mAP.
Limitations
- Performance depends on the quality of the MIL base model's proposals and snippet predictions.
- Several hyperparameters (alpha, beta, tau, lambda, learning rates) are tuned per dataset, including reuse of ActivityNet1.2 settings for ActivityNet1.3.
Experimental setup
Datasets: THUMOS14 ActivityNet1.3
Benchmarks: THUMOS14 ActivityNet1.3
Metrics: mAP@IoU average mAP
Baselines: CO2-Net DELU PMIL Li et al. PivoTAL DDG-Net ISSF A2Net TadTR AFormer TriDet
Builds on: DELU AFormer TriDet AutoLoc PivoTAL
Closest papers by meaning
- Improving Weakly Supervised Temporal Action Localization by Bridging Train-Test Gap in Pseudo Labels (CVPR 2023 · similarity 0.95)
- Rethinking Pseudo-Label Guided Learning for Weakly Supervised Temporal Action Localization from the Perspective of Noise Correction (AAAI 2025 · similarity 0.93)
- Similar Modality Enhancement and Action Consistency Learning for Weakly Supervised Temporal Action Localization (AAAI 2025 · similarity 0.91)
- Weakly-Supervised Temporal Action Localization by Inferring Salient Snippet-Feature (AAAI 2024 · similarity 0.91)
- HR-Pro: Point-Supervised Temporal Action Localization via Hierarchical Reliability Propagation (AAAI 2024 · similarity 0.90)
- Realigning Confidence with Temporal Saliency Information for Point-Level Weakly-Supervised Temporal Action Localization (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.