AI Paper Atlas

Research map / Label-efficient and robust learning / Knowledge distillation

DETRDistill: A Universal Knowledge Distillation Framework for DETR-families

Jiahao Chang, Shuo Wang, Hai-Ming Xu, Zehui Chen, Chenhongyi Yang, Feng Zhao

ICCV 2023 · Label-efficient and robust learning › Knowledge distillation · approach: Knowledge distillation for detectors · 52 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add negative-prediction box distillation to logits KD (ablation)DETRDistill logits distillation · COCO val2017 AdaMixerAP: 42.6 -> 43.6 (pos reg vs pos+neg reg)Improved
distill only positive predictions (ablation)logits KD · COCO val2017 AdaMixerAP: 42.3 -> 42.6No clear effect
distill stage-by-stage instead of teacher's last stage only (ablation)logits KD · COCO val2017 AdaMixerAP: 43.1 -> 43.7Improved
weight teacher queries by quality score in feature KD mask (ablation)target-aware feature distillation · COCO val2017 AdaMixerAP: 41.9 -> 43.5Improved
add query-prior assignment distillation to LD + FD (ablation)DETRDistill · COCO val2017 AdaMixerAP: 44.3 -> 44.7Improved
apply CNN feature distillation FGD to DETR studentAdaMixer · COCO val2017 12 epochsAP: 42.3 -> 40.7Hurt
apply DETRDistillDeformable DETR R50 · COCO val2017 50 epochsAP: 44.1 -> 46.6Improved
apply DETRDistillConditional DETR R50 · COCO val2017 50 epochsAP: 40.7 -> 42.9Improved
reduce decoder stages from 6 to 3 in student (ablation)Conditional DETR student · COCO val2017 12 epochsAP: 33.0 -> 30.3Hurt
reduce encoder stages from 6 to 3 in student (ablation)Conditional DETR student · COCO val2017 12 epochsAP: 33.0 -> 33.0No clear effect
apply DETRDistill to 3/3 stage studentConditional DETR student · COCO val2017 12 epochsAP: 29.9 -> 35.7Improved

Later papers that compared with this method

2 later accepted papers report DETRDistill as a baseline in their results tables; 2 of them report a better number on most of the results they share with it.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy