AI Paper Atlas

Research map / Detection and segmentation / CNN features

Masked Distillation with Receptive Tokens

Tao Huang, Yuan Zhang, Shan You, Fei Wang, Chen Qian, Jian Cao, Chang Xu

ICLR 2023 · Detection and segmentation › CNN features · approach: Local features and context in CNNs · 7 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Learn mask tokens with teacher task loss (vs. diversity loss alone) (ablation)MasKD mask learning · COCO, Faster RCNN-R50 student, Cascade Mask RCNN-X101 teacherAP: +0.6 AP over L_div-only (40.6 vs 40.0)Improved
Add mask diversity (Dice) loss to task-loss-learned tokens (ablation)MasKD mask learning · COCO, Faster RCNN-R50 student, Cascade Mask RCNN-X101 teacherAP: +0.4 AP (41.0 vs 40.6)Improved
Adaptive instance-dependent mask weighting (ablation)MasKD feature distillation · COCO, Faster RCNN-R50 student, Cascade Mask RCNN-X101 teacherAP: +0.2 AP (41.2 vs 41.0)Improved
Customize masks with student feature (ablation)MasKD feature distillation · COCO, Faster RCNN-R50 student, Cascade Mask RCNN-X101 teacherAP: +0.2 AP (41.4 vs 41.2)Improved
Full MasKD components vs. mimic baseline (ablation)Faster RCNN-R50 student · COCO, Cascade Mask RCNN-X101 teacherAP: +1.5 AP (41.4 vs 39.9)Improved
Random-initialized mask tokens with distillation (no learning) vs. L_div-only learned tokens (ablation)MasKD mask learning · COCO, Faster RCNN-R50 student, Cascade Mask RCNN-X101 teacherAP: L_div-only is 0.4 below random tokens (40.0 vs 40.4)Hurt
Increase number of receptive tokens from 2 to 14 (ablation)MasKD · COCO, Faster RCNN-R50 student, Mask RCNN-X101 teacherAP: 41.3 (2 tokens) to 41.5 (14 tokens); 6 tokens 41.4Improved
Use MasKD (learned receptive tokens) instead of Fitnet-style full mimic on detectionFaster RCNN-R50 student, R101 teacher · COCO val2017, 2x scheduleAP: 40.8 vs Fitnet 38.9 (+1.9 AP)Improved
Learned masks with student-customized distillation on segmentation (no weighting/customization used)DeepLabV3-R18 student, R101 teacher · Cityscapes test, from scratchmIoU (%): +6.69 over CIRKD (73.74 vs 68.22 on 0.5-width student)Improved
Ground-truth semantic masks vs. learned masks for distillation (ablation)DeepLabV3-R18 student, DeepLabV3-R101 teacher · Cityscapes valmIoU (%): gt 76.71 vs learned 77.00 (DeepLabV3); gt 75.26 vs learned 75.34 (PSPNet)No clear effect
Distill classification-head outputs in addition to features and regressionMasKD detection · COCOAPNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy