Research map / Detection and segmentation / CNN features
Masked Distillation with Receptive Tokens
Tao Huang, Yuan Zhang, Shan You, Fei Wang, Chen Qian, Jian Cao, Chang Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Learn mask tokens with teacher task loss (vs. diversity loss alone) (ablation) | MasKD mask learning · COCO, Faster RCNN-R50 student, Cascade Mask RCNN-X101 teacher | AP: +0.6 AP over L_div-only (40.6 vs 40.0) | Improved |
| Add mask diversity (Dice) loss to task-loss-learned tokens (ablation) | MasKD mask learning · COCO, Faster RCNN-R50 student, Cascade Mask RCNN-X101 teacher | AP: +0.4 AP (41.0 vs 40.6) | Improved |
| Adaptive instance-dependent mask weighting (ablation) | MasKD feature distillation · COCO, Faster RCNN-R50 student, Cascade Mask RCNN-X101 teacher | AP: +0.2 AP (41.2 vs 41.0) | Improved |
| Customize masks with student feature (ablation) | MasKD feature distillation · COCO, Faster RCNN-R50 student, Cascade Mask RCNN-X101 teacher | AP: +0.2 AP (41.4 vs 41.2) | Improved |
| Full MasKD components vs. mimic baseline (ablation) | Faster RCNN-R50 student · COCO, Cascade Mask RCNN-X101 teacher | AP: +1.5 AP (41.4 vs 39.9) | Improved |
| Random-initialized mask tokens with distillation (no learning) vs. L_div-only learned tokens (ablation) | MasKD mask learning · COCO, Faster RCNN-R50 student, Cascade Mask RCNN-X101 teacher | AP: L_div-only is 0.4 below random tokens (40.0 vs 40.4) | Hurt |
| Increase number of receptive tokens from 2 to 14 (ablation) | MasKD · COCO, Faster RCNN-R50 student, Mask RCNN-X101 teacher | AP: 41.3 (2 tokens) to 41.5 (14 tokens); 6 tokens 41.4 | Improved |
| Use MasKD (learned receptive tokens) instead of Fitnet-style full mimic on detection | Faster RCNN-R50 student, R101 teacher · COCO val2017, 2x schedule | AP: 40.8 vs Fitnet 38.9 (+1.9 AP) | Improved |
| Learned masks with student-customized distillation on segmentation (no weighting/customization used) | DeepLabV3-R18 student, R101 teacher · Cityscapes test, from scratch | mIoU (%): +6.69 over CIRKD (73.74 vs 68.22 on 0.5-width student) | Improved |
| Ground-truth semantic masks vs. learned masks for distillation (ablation) | DeepLabV3-R18 student, DeepLabV3-R101 teacher · Cityscapes val | mIoU (%): gt 76.71 vs learned 77.00 (DeepLabV3); gt 75.26 vs learned 75.34 (PSPNet) | No clear effect |
| Distill classification-head outputs in addition to features and regression | MasKD detection · COCO | AP | No clear effect |
Closest papers by meaning
- Masked Generative Distillation (ECCV 2022 · similarity 0.90)
- Progressive Mask Distillation for Self-supervised Video Representation (CVPR 2026 · similarity 0.89)
- Pixel-Wise Contrastive Distillation (ICCV 2023 · similarity 0.89)
- What to Hide from Your Students: Attention-Guided Masked Image Modeling (ECCV 2022 · similarity 0.88)
- DFD: Distilling the Feature Disparity Differently for Detectors (ICML 2024 · similarity 0.88)
- Hybrid Distillation: Connecting Masked Autoencoders with Contrastive Learners (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
