Research map / Video / Action recognition
Similar Modality Enhancement and Action Consistency Learning for Weakly Supervised Temporal Action Localization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SEAL, a weakly supervised temporal action localization method that builds hard-global descriptors from original RGB/flow features to generate pseudo-labels (Similar Modality Enhancement) and adds process and semantic consistency losses (Action Consistency Learning).
Problem
Weakly supervised temporal action localization with only video-level labels suffers from task discrepancy between frozen trimmed-action-classification encoders and localization, and from neglect of action consistency in process completeness and semantics.
Approach
Derive pseudo-labels directly from original features via appearance/motion global descriptors per class (cross-modal agreement gives high/moderate-confidence labels), plus a process consistency loss that penalizes proposals far from the action center and a semantic consistency loss that aligns proposal features to dynamically built soft-local descriptors.
Key results
- On THUMOS14, SEAL reaches 52.8% average mAP over IoU 0.1:0.7, a 3.2% gain over the best prior method (P-MIL/PivoTAL-level baselines at 49.6-47.0%).
- On ActivityNet1.3, SEAL achieves 25.6% average mAP over IoU 0.5:0.95, outperforming all compared methods on the average metric.
- Adding the SME and ACL losses to the base model raises average mAP from 43.1% to 52.8% on THUMOS14 (ablation at IoU 0.3/0.5/0.7).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the SME loss (ablation) | SEAL base model · THUMOS14 | AVG mAP: 43.1 -> 50.9 | Helped |
| add the process consistency loss (ablation) | SEAL base model · THUMOS14 | AVG mAP: 43.1 -> 51.2 | Helped |
| add the semantic consistency loss (ablation) | SEAL base model · THUMOS14 | AVG mAP: 43.1 -> 47.2 | Helped |
| add SME and process consistency losses (ablation) | SEAL base model · THUMOS14 | AVG mAP: 43.1 -> 52.1 | Helped |
| add SME and semantic consistency losses (ablation) | SEAL base model · THUMOS14 | AVG mAP: 43.1 -> 51.7 | Helped |
| add SME, process consistency, and semantic consistency losses (ablation) | SEAL base model · THUMOS14 | AVG mAP: 43.1 -> 52.8 | Helped |
| use video-level rather than segment-level aggregation for hard-global descriptors (ablation) | SME · THUMOS14 | AVG mAP: 50.8 -> 50.9 | No clear effect |
| add moderate-confidence pseudo-labels to high-confidence pseudo-labels (ablation) | SME · THUMOS14 | AVG mAP: 48.7 -> 50.9 | Helped |
| use P-WTAL candidate proposals with SEAL rather than the base model (ablation) | SEAL · THUMOS14 | AVG mAP: 43.1 -> 52.8 | Helped |
| use CO2-Net candidate proposals with SEAL rather than the base model (ablation) | SEAL · THUMOS14 | AVG mAP: 39.7 -> 47.1 | Helped |
| use SEAL rather than PivoTAL | THUMOS14 · THUMOS14 validation | mAP@0.1:0.7: 49.6 -> 52.8 | Helped |
| use SEAL rather than P-MIL | ActivityNet1.3 · ActivityNet1.3 validation | AVG mAP@0.5:0.95: 25.5 -> 25.6 | Helped |
Limitations
- Relies on raw proposals from LAC (point-level WTAL) without post-processing, so results depend on proposal source.
- Moderate-confidence pseudo-label weight decays and several hyperparameters (lambda1-3, alpha) need tuning.
- Gains on ActivityNet1.3 are smaller and SEAL does not top all methods at the 0.5 IoU threshold (41.6 vs 41.8 for P-MIL).
Experimental setup
Datasets: THUMOS14 ActivityNet1.3
Benchmarks: THUMOS14 ActivityNet1.3
Metrics: mAP@IoU average mAP
Baselines: STPN DGAM UM CoLA UGCT CO2-Net ASM-Loc DELU P-MIL PivoTAL AICL ISSF
Builds on: S-MIL P-MIL SCFE CO2-Net LAC
Closest papers by meaning
- PivoTAL: Prior-Driven Supervision for Weakly-Supervised Temporal Action Localization (CVPR 2023 · similarity 0.92)
- Actionness Inconsistency-Guided Contrastive Learning for Weakly-Supervised Temporal Action Localization (AAAI 2023 · similarity 0.92)
- Bridge the Gap: From Weak to Full Supervision for Temporal Action Localization with PseudoFormer (CVPR 2025 · similarity 0.91)
- Distilling Vision-Language Pre-Training To Collaborate With Weakly-Supervised Temporal Action Localization (CVPR 2023 · similarity 0.91)
- Weakly-Supervised Temporal Action Localization by Inferring Salient Snippet-Feature (AAAI 2024 · similarity 0.90)
- Weakly Supervised Temporal Action Localization via Dual-Prior Collaborative Learning Guided by Multimodal Large Language Models (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.