Research map / Label-efficient and robust learning / Knowledge distillation
Teach-DETR: Better Training DETR With Teachers
Linjiang Huang, Kaixin Lu, Guanglu Song, Liang Wang, Si Liu, Yu Liu, Hongsheng Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| distill features with FKD | H-Def-DETR student · MSCOCO val (H-Deformable-DETR Swin-S, 12 ep) | AP: 52.5 -> 52.0 | Hurt |
| distill features with FGD | H-Def-DETR student · MSCOCO val (H-Deformable-DETR Swin-S, 12 ep) | AP: 52.5 -> 51.7 | Hurt |
| add teacher box auxiliary supervision with scores | H-Def-DETR student · MSCOCO val (H-Deformable-DETR Swin-S, 12 ep) | AP: 52.5 -> 53.5 | Improved |
| concatenate GT and auxiliary boxes (ablation) | H-Def-DETR student · MSCOCO val (H-Deformable-DETR Swin-S, 12 ep) | AP: 52.5 -> 41.9 | Hurt |
| match GT and teacher boxes in parallel instead of concatenating (ablation) | Teach-DETR · MSCOCO val (H-Deformable-DETR Swin-S, 12 ep) | AP: 41.9 -> 53.1 | Improved |
| weight auxiliary supervision by teacher confidence scores (ablation) | Teach-DETR · MSCOCO val (H-Deformable-DETR Swin-S, 12 ep) | AP: 53.1 -> 53.5 | Improved |
| use teacher labels instead of GT labels (ablation) | Teach-DETR · MSCOCO val (H-Deformable-DETR Swin-S, 12 ep) | AP: 53.2 -> 53.5 | Improved |
| use teacher confidence scores instead of IoUs (ablation) | Teach-DETR · MSCOCO val (H-Deformable-DETR Swin-S, 12 ep) | AP: 53.3 -> 53.5 | Improved |
| use noisy boxes in place of teacher boxes (ablation) | H-Def-DETR student · MSCOCO val (H-Deformable-DETR Swin-S, 12 ep) | AP: 52.5 -> 51.9 | Hurt |
| use soft labels instead of hard labels (ablation) | Teach-DETR · MSCOCO val (H-Deformable-DETR Swin-S, 12 ep) | AP: 53.5 -> 46.8 | Hurt |
| use online instead of offline teachers (ablation) | Teach-DETR · MSCOCO val (H-Deformable-DETR Swin-S, 12 ep) | AP: 53.5 -> 53.6 | No clear effect |
| apply auxiliary supervision to all layers instead of final decoder layer only (ablation) | Teach-DETR · MSCOCO val (H-Deformable-DETR Swin-S, 12 ep) | AP: 52.7 -> 53.5 | Improved |
Closest papers by meaning
- DETRDistill: A Universal Knowledge Distillation Framework for DETR-families (ICCV 2023 · similarity 0.88)
- MS-DETR: Efficient DETR Training with Mixed Supervision (CVPR 2024 · similarity 0.88)
- DetKDS: Knowledge Distillation Search for Object Detectors (ICML 2024 · similarity 0.87)
- Mean Teacher DETR with Masked Feature Alignment: A Robust Domain Adaptive Detection Transformer Framework (AAAI 2024 · similarity 0.86)
- DETRs Beat YOLOs on Real-time Object Detection (CVPR 2024 · similarity 0.85)
- Hybrid Proposal Refiner: Revisiting DETR Series from the Faster R-CNN Perspective (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
