Research map / Video / Action recognition
Weakly-Supervised Temporal Action Localization by Inferring Salient Snippet-Feature
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ISSF, a weakly-supervised temporal action localization method that finds salient snippet-features via neighbor-snippet feature differences, refines them with channel- and temporal-level information interaction and a memory-based discrimination module, and uses the result as pseudo labels.
Problem
Weakly-supervised temporal action localization with only video-level labels; existing pseudo-label methods ignore the temporal structure of videos, making accurate snippet-level pseudo labels hard to generate.
Approach
Uses the L1 difference between temporally adjacent snippet-features to identify salient snippets (top-K), refines salient and non-salient features with squeeze-excitation-style channel interaction and temporal attention, and enhances discrimination with a class-wise memory bank; the refined features produce pseudo labels that supervise the base TCAM branch.
Key results
- Achieves 46.8 average mAP (IoU 0.1:0.7) on THUMOS14, and 57.8 at 0.1:0.5 and 37.6 at 0.3:0.7.
- Achieves 25.8 average mAP on ActivityNet v1.3, 0.7 higher than ASM-Loc.
- Ablation on THUMOS14: adding the boundary refinement module raises average mAP from 37.1 to 40.5, and adding the discrimination enhancement module raises it to 46.8.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed method rather than DELU | Weakly-supervised TAL · THUMOS14 test | average mAP@IoU (0.1:0.7): 46.4 -> 46.8 | Helped |
| Use the proposed method rather than ASM-Loc | Weakly-supervised TAL · ActivityNet v1.3 validation | average mAP: 25.1 -> 25.8 | Helped |
| Use L1 distance instead of random salient assignment (ablation) | Saliency inference module · THUMOS14 | average mAP: 42.3 -> 46.8 | Helped |
| Use L1 distance instead of classification-based salient selection (ablation) | Saliency inference module · THUMOS14 | average mAP: 43.0 -> 46.8 | Helped |
| Use L1 distance instead of cosine distance (ablation) | Saliency inference module · THUMOS14 | average mAP: 43.3 -> 46.8 | Helped |
| Use L1 distance instead of L2 distance (ablation) | Saliency inference module · THUMOS14 | average mAP: 44.2 -> 46.8 | Helped |
| Add the boundary refinement module (ablation) | Base branch · THUMOS14 | average mAP: 37.1 -> 40.5 | Helped |
| Add the discrimination enhancement module to Base + BRM (ablation) | Base + BRM · THUMOS14 | average mAP: 40.5 -> 46.8 | Helped |
| Use salient and non-salient features with weighted fusion instead of self interaction (ablation) | Boundary refinement module · THUMOS14 | average mAP: 39.2 -> 46.8 | Helped |
| Include salient snippet-features (ablation) | Boundary refinement module · THUMOS14 | average mAP: 27.6 -> 46.8 | Helped |
| Include non-salient snippet-features (ablation) | Boundary refinement module · THUMOS14 | average mAP: 45.9 -> 46.8 | Helped |
| Fuse salient and non-salient features with weighted sum instead of direct addition (ablation) | Boundary refinement module · THUMOS14 | average mAP: 43.1 -> 46.8 | Helped |
Surprising findings
- L1 distance outperforms L2 and cosine distance for detecting salient snippets, which the authors attribute to L2 suppressing subtle differences.
- Momentum-only memory update performs worse (44.5) than direct update (46.4), with the combined strategy best (46.8).
Limitations
- Relies on a fixed pre-trained I3D feature extractor and on the base classification head for pseudo labels.
- Performance at high IoU thresholds (0.6-0.7) on THUMOS14 is not uniformly best; the method lags some baselines there (e.g., DELU at 0.6 and 0.7).
Experimental setup
Datasets: THUMOS14 ActivityNet v1.3 Kinetics-400
Benchmarks: THUMOS14 ActivityNet v1.3
Metrics: mAP@IoU average mAP
Baselines: STPN RPN BaS-Net DGAM TSCN A2CL-PT UM CoLA AUMN UGCT D2-Net FAC-Net
Builds on: STPN BaS-Net CoLA ASM-Loc I3D
Closest papers by meaning
- Weakly Supervised Temporal Action Localization via Representative Snippet Knowledge Propagation (CVPR 2022 · similarity 0.94)
- CoLA: Weakly-Supervised Temporal Action Localization With Snippet Contrastive Learning (CVPR 2021 · similarity 0.94)
- Two-Stream Networks for Weakly-Supervised Temporal Action Localization With Semantic-Aware Mechanisms (CVPR 2023 · similarity 0.93)
- PivoTAL: Prior-Driven Supervision for Weakly-Supervised Temporal Action Localization (CVPR 2023 · similarity 0.92)
- Bridge the Gap: From Weak to Full Supervision for Temporal Action Localization with PseudoFormer (CVPR 2025 · similarity 0.91)
- Realigning Confidence with Temporal Saliency Information for Point-Level Weakly-Supervised Temporal Action Localization (CVPR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.