Research map / Video / Action recognition
Realigning Confidence with Temporal Saliency Information for Point-Level Weakly-Supervised Temporal Action Localization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TSP-Net, a plug-in proposal-level framework that realigns confidence for point-level weakly-supervised temporal action localization using a center score learned from saliency points, plus alignment-based boundary adaptation.
Problem
Point-level weakly-supervised temporal action localization suffers from misalignment between proposal quality and confidence scores, which are derived from unreliable class activation sequences without boundary information.
Approach
Exploits the observation that human point annotations tend to fall near instance centers (temporal saliency) to learn a center score per proposal supervised by a center label built from the points, updates the saliency points during training, fuses center scores with action probabilities into aligned confidence, and refines boundaries using aligned confidence and IoU among close proposals.
Key results
- On THUMOS'14, TSP-Net reaches 57.4% average mAP (0.1:0.7), 3.6% above CRRC-Net (53.8%).
- On GTEA and BEOID, average mAP (0.1:0.7) reaches 49.0% and 59.6%, improving over prior P-TAL state of the art by 4.1% and 6.3%.
- Plugging TSP-Net into SF-Net, BackTAL and LAC yields an average improvement of 5.8% on THUMOS'14.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply TSP-Net to LAC | LAC · THUMOS’14 | mAP@IoU (0.1:0.7): 52.8 -> 57.4 | Helped |
| Apply TSP-Net to LAC | LAC · GTEA | mAP@IoU (0.1:0.7): 43.5 -> 49.0 | Helped |
| Apply TSP-Net to LAC | LAC · BEOID | mAP@IoU (0.1:0.7): 51.8 -> 59.6 | Helped |
| Add proposal learning to the baseline (ablation) | TSP-Net · THUMOS’14 | mAP@IoU (0.1:0.7): 52.5 -> 55.0 | Helped |
| Add center label generation (ablation) | TSP-Net · THUMOS’14 | mAP@IoU (0.1:0.7): 55.0 -> 55.7 | Helped |
| Add saliency point updating (ablation) | TSP-Net · THUMOS’14 | mAP@IoU (0.1:0.7): 56.5 -> 56.9 | Helped |
| Add alignment-based boundary adaptation (ablation) | TSP-Net · THUMOS’14 | mAP@IoU (0.1:0.7): 56.9 -> 57.4 | Helped |
| Use hard center labels instead of soft labels (ablation) | TSP-Net · THUMOS’14 | mAP@IoU (0.1:0.7): 57.4 -> 56.2 | Hurt |
| Set the soft-label value λ to 0.2 rather than 0.4 (ablation) | TSP-Net · THUMOS’14 | mAP@IoU (0.1:0.7): 57.4 -> 56.6 | Hurt |
| Set the soft-label value λ to 0.3 rather than 0.4 (ablation) | TSP-Net · THUMOS’14 | mAP@IoU (0.1:0.7): 57.4 -> 57.1 | Hurt |
| Set the soft-label value λ to 0.5 rather than 0.4 (ablation) | TSP-Net · THUMOS’14 | mAP@IoU (0.1:0.7): 57.4 -> 56.8 | Hurt |
| Use iterative saliency point updating instead of SPU (ablation) | TSP-Net · THUMOS’14 | mAP@IoU (0.1:0.7): 57.4 -> 56.6 | Hurt |
Surprising findings
- Placing all point annotations at instance boundaries makes the method fail, while absolute-center points are suboptimal, suggesting that some noise in annotation placement helps.
- Iteration-based saliency point updating accumulates noise and underperforms updating from the initial points.
Limitations
- Gains are smaller for SF-Net because it generates far fewer proposals without multi-threshold generation.
- Relies on a CAS-based base locator (LAC) for proposal generation, so proposal quality is bounded by the base locator.
Experimental setup
Datasets: THUMOS'14 BEOID GTEA
Benchmarks: THUMOS'14 GTEA BEOID
Metrics: mAP@IoU average mAP
Baselines: LAC CRRC-Net PCL DCM BackTAL SF-Net P-MIL PivoTAL
Builds on: LAC P-MIL SF-Net Soft-NMS I3D
Closest papers by meaning
- Weakly-Supervised Temporal Action Localization by Inferring Salient Snippet-Feature (AAAI 2024 · similarity 0.91)
- Bridge the Gap: From Weak to Full Supervision for Temporal Action Localization with PseudoFormer (CVPR 2025 · similarity 0.90)
- HR-Pro: Point-Supervised Temporal Action Localization via Hierarchical Reliability Propagation (AAAI 2024 · similarity 0.90)
- Test-Time Zero-Shot Temporal Action Localization (CVPR 2024 · similarity 0.89)
- Energy vs. Noise: Towards Robust Temporal Action Localization in Open-World (AAAI 2025 · similarity 0.88)
- Temporal Action Localization with Cross Layer Task Decoupling and Refinement (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.