Research map / Video / Action recognition
Generative Model-Based Feature Knowledge Distillation for Action Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a generative-model (conditional VAE) based feature knowledge distillation framework that transfers attention-conditioned feature semantics from a 3D-CNN teacher (I3D) to a lightweight student (Top-I3D) for video action recognition and detection.
Problem
Compressing 3D-CNN video models via knowledge distillation: existing video KD methods focus on loss design and cross-modal fusion and neglect spatial-temporal feature semantics.
Approach
Uses a conditional VAE to model feature distributions conditioned on an attention map; alternates a Generative Distillation stage (matching CVAE-reconstructed features) with an Attention Distillation stage (matching attention maps plus reconstruction and classification losses), so only the student plus attention module is used at inference.
Key results
- On UCF101, Top-I3D student goes from 64.1% to 66.6% Top-1 (teacher I3D 91.9%), outperforming FN (65.5%) and SimKD (66.1%).
- On HMDB51, the student improves from 52.0% to 54.5% Top-1.
- On THUMOS14 action detection, the AFSD student average mAP rises from 32.6% to 33.7%, at 39.9 GFLOPs vs teacher 84.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add feature representation loss L_recon (ablation) | Top-I3D classification baseline · UCF101 | Top-1 accuracy: 64.1 -> 65.2 | Helped |
| add generative distillation loss L_KD-cvae after L_recon (ablation) | Top-I3D classification baseline · UCF101 | Top-1 accuracy: 65.2 -> 66.2 | Helped |
| add attention distillation loss L_KD-att after L_recon (ablation) | Top-I3D classification baseline · UCF101 | Top-1 accuracy: 65.2 -> 66.1 | Helped |
| add both generative and attention distillation losses (ablation) | Top-I3D classification baseline with L_recon · UCF101 | Top-1 accuracy: 64.1 -> 66.6 | Helped |
| add attention-based feature representation module (ablation) | I3D · UCF101 | Top-1 accuracy: 91.9 -> 92.4 | Helped |
| add attention-based feature representation module (ablation) | Top-I3D · UCF101 | Top-1 accuracy: 64.1 -> 64.7 | Helped |
| apply the full KD framework (ablation) | Top-I3D · UCF101 | Top-1 accuracy: 64.1 -> 66.2 | Helped |
| apply the full KD framework (ablation) | Bottom-I3D · UCF101 | Top-1 accuracy: 52.7 -> 53.3 | Helped |
| apply the full KD framework (ablation) | I2D · UCF101 | Top-1 accuracy: 52.2 -> 58.6 | Helped |
| use generative model-based feature knowledge distillation | Top-I3D student with I3D teacher · UCF101 | Top-1 accuracy: 64.1 -> 66.6 | Helped |
| use generative model-based feature knowledge distillation | Top-I3D student with I3D teacher · HMDB51 | Top-1 accuracy: 52.0 -> 54.5 | Helped |
| use generative model-based feature knowledge distillation | Top-I3D-based AFSD student with AFSD teacher · THUMOS14 | AVG mAP: 32.6 -> 33.7 | Helped |
Surprising findings
- The student's FLOPs increase slightly (45.5G to 47.4G in recognition; 36.0G to 39.9G in detection) with the distillation framework, which the authors describe as a marginal overhead.
Limitations
- Results are reported on validation/test splits of three datasets with a single teacher-student pair family.
- Student still shows a clear accuracy gap relative to the teacher (e.g., 66.6% vs 91.9% on UCF101).
- Gains on Bottom-I3D are small (0.6% Top-1).
Experimental setup
Datasets: UCF101 HMDB51 THUMOS14
Benchmarks: UCF101 HMDB51 THUMOS14
Metrics: Top-1 accuracy Top-5 accuracy mAP at tIoU thresholds FLOPs
Baselines: KD CTKD FN MGD SimKD AT AFSD
Builds on: feature knowledge distillation attention transfer conditional VAE I3D
Closest papers by meaning
- Knowledge Distillation As Efficient Pre-Training: Faster Convergence, Higher Data-Efficiency, and Better Transferability (CVPR 2022 · similarity 0.88)
- Improve Object Detection with Feature-based Knowledge Distillation: Towards Accurate and Efficient Detectors (ICLR 2021 · similarity 0.88)
- VkD: Improving Knowledge Distillation using Orthogonal Projections (CVPR 2024 · similarity 0.87)
- DCSF-KD: Dynamic Channel-wise Spatial Feature Knowledge Distillation for Object Detection (AAAI 2025 · similarity 0.86)
- DFD: Distilling the Feature Disparity Differently for Detectors (ICML 2024 · similarity 0.86)
- Why Knowledge Distillation Works in Generative Models: A Minimal Working Explanation (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.