Research map / Video / Action recognition
Semi-supervised Active Learning for Video Action Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a semi-supervised active learning framework for video action detection that combines mean-teacher SSL with FFT-attention (high-pass filter weighting of consistency loss) and NoiseAug (noise-variance-based sample selection).
Problem
Label-efficient video action detection, where spatio-temporal annotation is costly; active learning suffers cold start and SSL relies on random sampling and noisy background pseudo-labels.
Approach
Uses noise-augmented variants of each unlabeled video and the variance of temporally averaged uncertainty as an informativeness score (NoiseAug), and an FFT-based high-pass filter that weights the mean-teacher consistency loss toward action regions and edges (fft-attention).
Key results
- On UCF101-24 with 20% labels, M-T SSL + AL reaches 72.0 f-mAP and 74.5 v-mAP@0.5, versus 59.8 f-mAP for the supervised baseline.
- On JHMDB-21 with 30% labels, M-T SSL + AL reaches 71.7 v-mAP@0.5, reported as +8.2 over the prior best.
- On YouTube-VOS with 20% labels, the method reaches 41.6 average J&F versus 34.7 for random selection.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add NoiseAug-based active learning selection to mean-teacher SSL (ablation) | VideoCapsuleNet · UCF101-24, 20% labels | v-mAP@0.5: 72.2 -> 74.5 | Helped |
| add NoiseAug-based active learning selection to mean-teacher SSL (ablation) | VideoCapsuleNet · JHMDB-21, 30% labels | v-mAP@0.5: 68.0 -> 71.7 | Helped |
| add FFT high-pass filtering to mean-teacher consistency loss (ablation) | Mean-teacher SSL · UCF101-24, 10% labels | f-mAP@0.5: 67.2 -> 68.5 | Helped |
| add FFT high-pass filtering to mean-teacher consistency loss (ablation) | Mean-teacher SSL · JHMDB-21, 20% labels | f-mAP@0.5: 61.5 -> 67.1 | Helped |
| use mean-teacher SSL instead of consistency-based SSL (ablation) | VideoCapsuleNet · UCF101-24, 10% labels | f-mAP@0.5: 62.4 -> 67.2 | Helped |
| use mean-teacher SSL instead of consistency-based SSL (ablation) | VideoCapsuleNet · JHMDB-21, 20% labels | f-mAP@0.5: 62.6 -> 61.5 | Hurt |
| replace NoiseAug selection with mean-teacher strong/weak augmentations (ablation) | Active learning selection for VideoCapsuleNet · UCF101-24, 10% labeled data | v-mAP@0.5: 62.3 -> 70.4 | Helped |
| add the full proposed method instead of PI-consistency | Video action segmentation · YouTube-VOS, 10% data | Avg: 36.8 -> 39.3 | Helped |
| use the proposed method instead of random selection | Video action segmentation · YouTube-VOS, 20% data | Avg: 34.7 -> 41.6 | Helped |
Surprising findings
- Using the same strong/weak augmentations as SSL for AL selection gives no gain over random selection at 5-20% labels, whereas NoiseAug gives large gains.
Limitations
- Evaluated only on action detection datasets with a single backbone (VideoCapsuleNet).
- Hyperparameters such as the FFT radius and loss weights are tuned per setting and the method is reported as sensitive to filter radius beyond a range.
Experimental setup
Datasets: UCF101-24 JHMDB-21 YouTube-VOS
Benchmarks: UCF101-24 JHMDB-21 YouTube-VOS
Metrics: f-mAP v-mAP@0.2 v-mAP@0.5 J&F (average IoU and boundary similarity)
Baselines: MixMatch Pseudo-label Co-SSD (CC) PI-consistency MC uncertainty MC entropy random selection Kalogeitan et al. Song et al. Li et al. Gu et al. Duarte et al.
Builds on: Mean Teacher VideoCapsuleNet PI-consistency
Closest papers by meaning
- End-to-End Semi-Supervised Learning for Video Action Detection (CVPR 2022 · similarity 0.92)
- Stable Mean Teacher for Semi-supervised Video Action Detection (AAAI 2025 · similarity 0.90)
- Are all Frames Equal? Active Sparse Labeling for Video Action Detection (NeurIPS 2022 · similarity 0.90)
- Hybrid Active Learning via Deep Clustering for Video Action Detection (CVPR 2023 · similarity 0.89)
- Large-Scale Weakly-Supervised Pre-Training for Video Action Recognition (CVPR 2019 · similarity 0.88)
- Rethinking Pseudo-Label Guided Learning for Weakly Supervised Temporal Action Localization from the Perspective of Noise Correction (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.