Research map / Label-efficient and robust learning / Knowledge distillation
Evidential Knowledge Distillation
Liangyu Xiang, Junyu Gao, Changsheng Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace vanilla KD loss with first-order Dirichlet-expectation loss L1st alongside L2nd (ablation) | EKD (ResNet32x4->ResNet8x4) · CIFAR-100 val (ablation) | top-1 accuracy: 76.34 -> 77.21 | Improved |
| replace MSE magnitude alignment with second-order Dirichlet KL L2nd alongside L1st (ablation) | EKD (ResNet32x4->ResNet8x4) · CIFAR-100 val (ablation) | top-1 accuracy: 76.83 -> 77.21 | Improved |
| replace MSE with L2nd when paired with vanilla KD (ablation) | KD (ResNet32x4->ResNet8x4) · CIFAR-100 val (ablation) | top-1 accuracy: 76.13 -> 76.34 | Improved |
| replace vanilla KD with L1st when paired with MSE (ablation) | KD+MSE (ResNet32x4->ResNet8x4) · CIFAR-100 val (ablation) | top-1 accuracy: 76.13 -> 76.83 | Improved |
| train teacher and student with evidential loss (EDL) instead of softmax cross-entropy | ResNet8x4 student · CIFAR-100 val | top-1 accuracy: 72.50 -> 72.77 | No clear effect |
| train MobileNetV2 student with EDL instead of softmax | MN-V2 student · CIFAR-100 val | top-1 accuracy: 64.60 -> 65.39 | Improved |
| replace vanilla KD with EKD (ResNet32x4->ResNet8x4) | ResNet8x4 student · CIFAR-100 val | top-1 accuracy: 73.33 -> 77.21 | Improved |
| replace vanilla KD with EKD on ResNet50->MobileNetV1 | MobileNetV1 student · ImageNet val | top-1 accuracy: 70.50 -> 72.54 | Improved |
| replace vanilla KD with EKD on ResNet34->ResNet18 | ResNet18 student · ImageNet val | top-1 accuracy: 71.03 -> 71.73 | Improved |
| add EKD to RKD feature distillation (VGG13->VGG8) | RKD · CIFAR-100 val | top-1 accuracy: 71.48 -> 74.99 | Improved |
| add EKD to RKD feature distillation (ResNet32x4->SHN-V2) | RKD · CIFAR-100 val | top-1 accuracy: 73.21 -> 78.08 | Improved |
| add EKD to CAT-KD (VGG13->VGG8) | CAT-KD · CIFAR-100 val | top-1 accuracy: 74.65 -> 74.91 | Improved |
Closest papers by meaning
- SGD-Based Knowledge Distillation with Bayesian Teachers: Theory and Guidelines (ICLR 2026 · similarity 0.90)
- Bayesian Knowledge Distillation: A Bayesian Perspective of Distillation with Uncertainty Quantification (ICML 2024 · similarity 0.88)
- Debiased Distillation for Consistency Regularization (AAAI 2025 · similarity 0.87)
- PLD: A Choice-Theoretic List-Wise Knowledge Distillation (NeurIPS 2025 · similarity 0.87)
- ABKD: Pursuing a Proper Allocation of the Probability Mass in Knowledge Distillation via $\alpha$-$\beta$-Divergence (ICML 2025 · similarity 0.87)
- Knowledge Distillation with Auxiliary Variable (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
