Research map / Label-efficient and robust learning / Knowledge distillation
Perspective-Aware Teaching: Adapting Knowledge for Heterogeneous Distillation
Jhe-Hao Lin, Yi Yao, Chan-Feng Hsu, Hong-Xia Xie, Hong-Han Shuai, Wen-Huang Cheng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add region-aware attention (RAA) to FitNet (ablation) | FitNet · CIFAR-100 (ConvNeXt-T to DeiT-T) | accuracy: 60.71 -> 70.12 | Improved |
| add AFP without feedback on top of RAA (ablation) | PAT · CIFAR-100 (ConvNeXt-T to DeiT-T) | accuracy: 70.12 -> 79.13 | Improved |
| add student feedback into AFP (ablation) | PAT · CIFAR-100 (ConvNeXt-T to DeiT-T) | accuracy: 79.13 -> 79.59 | Improved |
| apply AFP at stage 4 only versus stage 2 only (ablation) | PAT · CIFAR-100 (ConvNeXt-T to DeiT-T) | accuracy: 69.65 vs 77.34 | Mixed |
| apply AFP at all four stages instead of stages 2-4 (ablation) | PAT · CIFAR-100 (ConvNeXt-T to DeiT-T) | accuracy: 78.71 -> 79.59 | Improved |
| increase RAA queries N_q from 36 to 144 (ablation) | PAT · CIFAR-100 (ConvNeXt-T to DeiT-T) | accuracy: 79.03 -> 80.89 | Improved |
| add KL loss to PAT (ConvNeXt-T to ResMLP-S12) (ablation) | PAT · CIFAR-100 (ConvNeXt-T to ResMLP-S12) | accuracy: 81.79 -> 83.50 | Improved |
| add KL loss to PAT (ConvNeXt-T to DeiT-T) (ablation) | PAT · CIFAR-100 (ConvNeXt-T to DeiT-T) | accuracy: 75.98 -> 79.10 | Improved |
| use PAT instead of OFA-KD | PAT · CIFAR-100 (12 heterogeneous pairs) | average improvement: 7.47 -> 8.17 | Improved |
| use PAT instead of OFA-KD on detection | PAT · COCO (Swin-T to ResNet18) | mAP: 33.37 -> 35.62 | Improved |
| use PAT instead of OFA-KD with MobileNetV2 student | PAT · COCO (Swin-T to MobileNetV2) | mAP: 31.69 -> 32.97 | Improved |
| use PAT instead of OFA-KD with ViT-S to ResNet18 | PAT · CIFAR-100 (ViT-S to ResNet18) | accuracy: 80.15 -> 80.11 | No clear effect |
Closest papers by meaning
- Heterogeneous Complementary Distillation (AAAI 2026 · similarity 0.92)
- Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks (TPAMI 2021 · similarity 0.91)
- Single-Teacher View Augmentation: Boosting Knowledge Distillation via Angular Diversity (NeurIPS 2025 · similarity 0.91)
- All You Need in Knowledge Distillation Is a Tailored Coordinate System (AAAI 2025 · similarity 0.89)
- Neural Collapse Inspired Knowledge Distillation (AAAI 2025 · similarity 0.89)
- Distilling Knowledge from Heterogeneous Architectures for Semantic Segmentation (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
