Research map / Video / Action recognition
Stable Mean Teacher for Semi-supervised Video Action Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Stable Mean Teacher, a student-teacher SSL framework for video action detection with an Error Recovery module and a Difference-of-Pixels temporal consistency loss, evaluated on UCF101-24, JHMDB21, AVA and YouTube-VOS.
Problem
Semi-supervised spatio-temporal video action detection suffers from unreliable pseudo-labels under limited annotations, and existing spatio-temporal losses ignore temporal coherence across frames.
Approach
Adapts mean teacher to spatio-temporal localization with a class-agnostic Error Recovery module that learns from student mistakes on labeled data and refines teacher pseudo-labels, plus a Difference-of-Pixels loss that matches frame-to-frame pixel differences to enforce temporal coherence.
Key results
- On UCF101-24 with 10% labels, Stable Mean Teacher reaches 73.9 f-mAP@0.5 and 76.3 v-mAP@0.5, versus 53.5 and 49.7 for the supervised baseline on the same labeled subset.
- On JHMDB21 with 20% labels, it reaches 69.8 f-mAP@0.5 and 70.7 v-mAP@0.5, improving over the 20%-supervised baseline (55.7 / 52.4).
- On AVA with 10% labels, it improves YOWOv2-N from 5.2 to 8.5 mAP; on YouTube-VOS at 10% labels it gets avg 41.3 vs 36.8 for Kumar et al.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the EoR module (ablation) | STMT · UCF101-24 | v-mAP@0.5: 74.5 -> 75.9 | Helped |
| add the EoR module (ablation) | STMT · JHMDB21 | v-mAP@0.5: 62.0 -> 68.1 | Helped |
| add the DoP constraint (ablation) | STMT · UCF101-24 | v-mAP@0.5: 74.5 -> 75.4 | Helped |
| add the DoP constraint (ablation) | STMT · JHMDB21 | v-mAP@0.5: 62.0 -> 64.5 | Helped |
| add EoR and DoP (ablation) | STMT · JHMDB21 | f-mAP@0.5: 61.8 -> 69.8 | Helped |
| use 2D instead of 3D for the EoR architecture (ablation) | Stable Mean Teacher · UCF101-24 | f-mAP@0.5: 73.9 -> 73.6 | Hurt |
| allow EoR gradients to update the main network (ablation) | Stable Mean Teacher · video action detection | performance: approximately 3% | Hurt |
| add EoR parameters directly to the base model instead of using EoR refinement (ablation) | STMT · JHMDB21 (20% labels) | f-mAP@0.5: 61.8 -> 62.3 | Mixed |
| use burn-in weights before combined labeled and unlabeled training (ablation) | Stable Mean Teacher · video action detection | performance | No clear effect |
| use Stable Mean Teacher instead of Baseline Mean Teacher | I3D · UCF101-24 (10% labels) | f-mAP@0.5: 67.3 -> 73.9 | Helped |
| use Stable Mean Teacher instead of Baseline Mean Teacher | I3D · JHMDB21 (20% labels) | f-mAP@0.5: 56.3 -> 69.8 | Helped |
| use semi-supervised Stable Mean Teacher instead of the supervised baseline | YOWOv2-N · AVA (10% labels) | mAP: 5.2 -> 8.5 | Helped |
Surprising findings
- Adding the EoR module's parameters to the base model without the gradient-stopping design gives only about 0.5-1% gain, so the improvement is not simply due to extra capacity.
- Burn-in pre-training on the labeled set gave no substantial gain over end-to-end training.
Limitations
- Labeled/unlabeled splits and reduced label percentages (10%/20%) differ from prior protocols, complicating direct comparison.
- Gains on AVA are modest (3.3%) and the approach relies on a lightweight base model (VideoCapsuleNet 2D routing).
Experimental setup
Datasets: UCF101-24 JHMDB21 AVA v2.2 YouTube-VOS 2019
Benchmarks: UCF101-24 JHMDB21 AVA v2.2 YouTube-VOS 2019
Metrics: f-mAP v-mAP Jaccard (J) boundary (F) mAP
Baselines: Mean Teacher Pseudo-label ISD E2E-SSL MixMatch Kumar et al. (PI-consistency video action detection) Supervised baseline YOWOv2-N
Builds on: Mean Teacher VideoCapsuleNet 3D UNet
Closest papers by meaning
- End-to-End Semi-Supervised Learning for Video Action Detection (CVPR 2022 · similarity 0.93)
- Semi-supervised Active Learning for Video Action Detection (AAAI 2024 · similarity 0.90)
- TimeBalance: Temporally-Invariant and Temporally-Distinctive Video Representations for Semi-Supervised Action Recognition (CVPR 2023 · similarity 0.90)
- STEP: Spatio-Temporal Progressive Learning for Video Action Detection (CVPR 2019 · similarity 0.88)
- A flexible model for training action localization with varying levels of supervision (NeurIPS 2018 · similarity 0.88)
- Rethinking Pseudo-Label Guided Learning for Weakly Supervised Temporal Action Localization from the Perspective of Noise Correction (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.