Research map / Video / Action recognition
Energy vs. Noise: Towards Robust Temporal Action Localization in Open-World
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EDMP, a plug-and-play meta-learning method with energy-based semantic purification, temporal boundary refinement, and loss reweighting that makes temporal action localization models robust to boundary and category label noise.
Problem
Temporal action localization models degrade sharply when training annotations contain boundary noise and category noise from crowdsourced labeling, limiting open-world use.
Approach
Models the foreground/background difference as a temporal energy (Helmholtz free energy over logits) and uses it inside a meta-learning loop to correct category labels (SPM), refine boundaries with a boundary energy loss (TRM), and reweight sample losses (ESO), using a small clean meta set of one video per class.
Key results
- On THUMOS14 under mixed noise 0.5x50%, EDMP raises ActionFormer average mAP by 5.79%, TriDet by 5.33%, and TemporalMaxer by 6.43%.
- Ablation on THUMOS14 (ActionFormer, mixed noise 0.5x50%): the full model reaches 50.05 average mAP versus 44.26 for the baseline.
- Compared with LNL methods on THUMOS14 (mixed 0.3x30%), EDMP reaches 58.35 average mAP, above DMLP (57.59), MLC (57.32), MWN (57.13) and Co-teaching (54.87).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add EDMP to ActionFormer | ActionFormer · THUMOS14, mixed noise 0.3×30% | average mAP (%): 53.95 -> 58.35 | Helped |
| Add EDMP to TriDet | TriDet · THUMOS14, mixed noise 0.3×30% | average mAP (%): 57.64 -> 60.58 | Helped |
| Add EDMP to TemporalMaxer | TemporalMaxer · THUMOS14, mixed noise 0.3×30% | average mAP (%): 58.21 -> 62.90 | Helped |
| Add EDMP to ActionFormer | ActionFormer · ActivityNet1.3, mixed noise 0.3×30% | average mAP (%): 25.32 -> 29.42 | Helped |
| Add EDMP to TriDet | TriDet · ActivityNet1.3, mixed noise 0.3×30% | average mAP (%): 25.85 -> 29.76 | Helped |
| Add EDMP to TemporalMaxer | TemporalMaxer · ActivityNet1.3, mixed noise 0.3×30% | average mAP (%): 24.87 -> 28.97 | Helped |
| Add EDMP to TemporalMaxer | TemporalMaxer · THUMOS14, no noise | average mAP (%): 67.16 -> 67.09 | No clear effect |
| Add EDMP to ActionFormer | ActionFormer · ActivityNet1.3, no noise | average mAP (%): 36.38 -> 36.37 | No clear effect |
| Add Meta-learning Pipeline (ablation) | EDMP · THUMOS14, mixed noise 0.5×50% | average mAP (%): 44.26 -> 50.05 | Helped |
| Add SPM (ablation) | EDMP with Meta-learning Pipeline · THUMOS14, mixed noise 0.5×50% | average mAP (%): 45.23 -> 47.74 | Helped |
| Add TRM (ablation) | EDMP with Meta-learning Pipeline · THUMOS14, mixed noise 0.5×50% | average mAP (%): 45.23 -> 47.12 | Helped |
| Add ESO (ablation) | EDMP with Meta-learning Pipeline, SPM, and TRM · THUMOS14, mixed noise 0.5×50% | average mAP (%): 48.54 -> 50.05 | Helped |
Surprising findings
- Even small mixed noise (0.1x10%) drops leading TAL models by 3.55-5.16% mAP, showing strong sensitivity to label noise.
Limitations
- Requires a small clean meta set (one accurately annotated video per class), which still needs precise annotation.
- Noise is simulated synthetically (Gaussian boundary jitter and confusion-group category swaps) rather than from real annotator noise.
Experimental setup
Datasets: THUMOS14 ActivityNet1.3
Benchmarks: THUMOS14 ActivityNet1.3
Metrics: mAP at tIoU thresholds average mAP
Baselines: ActionFormer TriDet TemporalMaxer Co-teaching MWN MLC DMLP
Builds on: ActionFormer TriDet TemporalMaxer Co-teaching Helmholtz free energy
Closest papers by meaning
- METAL: Minimum Effort Temporal Activity Localization in Untrimmed Videos (CVPR 2020 · similarity 0.89)
- Realigning Confidence with Temporal Saliency Information for Point-Level Weakly-Supervised Temporal Action Localization (CVPR 2024 · similarity 0.88)
- Temporal Action Localization with Cross Layer Task Decoupling and Refinement (AAAI 2025 · similarity 0.88)
- Bridge the Gap: From Weak to Full Supervision for Temporal Action Localization with PseudoFormer (CVPR 2025 · similarity 0.88)
- Test-Time Zero-Shot Temporal Action Localization (CVPR 2024 · similarity 0.87)
- Weakly-Supervised Temporal Action Localization by Inferring Salient Snippet-Feature (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.