Research map / Label-efficient and robust learning / Knowledge distillation
Delta Distillation for Efficient Video Processing
Amirhossein Habibian, Haitam Ben Yahia, Davide Abati, Efstratios Gavves, Fatih Porikli
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add delta distillation to cut FLOPs | EfficientDet-D0 · ImageNet VID | GFLOPs: 2.50 -> 0.71 | Improved |
| add delta distillation (at 0.71 GFLOPs) | EfficientDet-D0 · ImageNet VID | mAP: 69.0 -> 66.1 | Hurt |
| add delta distillation at moderate compression | EfficientDet-D0 · ImageNet VID | mAP: 69.0 -> 68.8 | No clear effect |
| add delta distillation | Faster-RCNN R101 · ImageNet VID | GFLOPs: 149.1 -> 29.2 | Improved |
| add delta distillation | Faster-RCNN R101 · ImageNet VID | mAP: 74.5 -> 73.5 | Hurt |
| add delta distillation | DDRNet-39 · Cityscapes val | mIoU: 79.5 -> 79.9 | Improved |
| add delta distillation | DDRNet-23-slim · Cityscapes val | mIoU: 76.1 -> 76.2 | No clear effect |
| add delta distillation | HRNet-w18-small · Cityscapes val | mIoU: 76.1 -> 75.7 | Hurt |
| distill deltas instead of features in student (ablation) | EfficientDet-D0 · ImageNet VID | GFLOPs: 2.5 -> 1.14 | Improved |
| train with delta distillation (teacher only) | DDRNet23 · Cityscapes val | temporal consistency: 82.6 -> 84.4 | Improved |
| use teacher and student at inference (T+S) rather than teacher only (ablation) | DDRNet23 + delta distillation · Cityscapes val | temporal consistency: 84.4 -> 85.2 | Improved |
| use linear students instead of non-linear channel-reduction students (ablation) | DDRNet23 delta distillation · Cityscapes val | GFLOPs: 84.3 -> 71.8 | Improved |
Closest papers by meaning
- Online Model Distillation for Efficient Video Inference (ICCV 2019 · similarity 0.90)
- Scaling Video-Language Models to 10K Frames via Hierarchical Differential Distillation (ICML 2025 · similarity 0.88)
- DFD: Distilling the Feature Disparity Differently for Detectors (ICML 2024 · similarity 0.87)
- Dancing with Still Images: Video Distillation via Static-Dynamic Disentanglement (CVPR 2024 · similarity 0.87)
- Distilling Knowledge from Heterogeneous Architectures for Semantic Segmentation (AAAI 2025 · similarity 0.87)
- Understanding the Role of the Projector in Knowledge Distillation (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
