Research map / Adversarial robustness and security / Certified and empirical robustness
NEO-KD: Knowledge-Distillation-Based Adversarial Training for Robust Multi-Exit Neural Networks
Seokil Ham, Jungwuk Park, Dong-Jun Han, Jaekyun Moon
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| NEO-KD instead of plain adversarial training | MSDNet · CIFAR-10 max-average | adv. accuracy (avg): 45.91% -> 48.30% | Improved |
| NEO-KD instead of plain adversarial training | MSDNet · CIFAR-10 average attack | adv. accuracy (avg): 40.04% -> 44.20% | Improved |
| NEO-KD instead of plain adversarial training | MSDNet · CIFAR-100 average attack | top-1 adv. accuracy: 18.13 -> 22.88 | Improved |
| NEO-KD instead of plain adversarial training | MSDNet · Tiny-ImageNet average attack | top-1 adv. accuracy: 27.61 -> 30.56 | Improved |
| NEO-KD instead of plain adversarial training | MSDNet · ImageNet average attack | top-1 adv. accuracy: 25.58 -> 29.98 | Improved |
| NEO-KD instead of plain adversarial training | SmallCNN · MNIST max-average | adv. accuracy (avg): 94.15% -> 94.49% | Improved |
| add self-knowledge distillation (SKD) | adversarial training · CIFAR-10 max-average | adv. accuracy (avg): 45.91% -> 44.69% | Hurt |
| add LW regularizer | adversarial training · CIFAR-10 max-average | adv. accuracy (avg): 45.91% -> 35.87% | Hurt |
| add only NKD (remove EOKD) (ablation) | NEO-KD · CIFAR-10 average attack | adv. accuracy (avg): 40.04% -> 40.97% | Improved |
| add only EOKD (remove NKD) (ablation) | NEO-KD · CIFAR-10 average attack | adv. accuracy (avg): 40.04% -> 39.01% | Hurt |
| ensemble neighbor exits instead of no ensembling (ablation) | NKD · CIFAR-10 average attack | adv. accuracy (avg): 40.49% -> 44.20% | Improved |
| ensemble all exits instead of neighbors (ablation) | NKD · CIFAR-10 average attack | adv. accuracy (avg): 44.20% -> 40.06% | Hurt |
Closest papers by meaning
- Rethinking the Adversarial Robustness of Multi-Exit Neural Networks in an Attack-Defense Game (CVPR 2025 · similarity 0.88)
- Robust Distillation via Untargeted and Targeted Intermediate Adversarial Samples (CVPR 2024 · similarity 0.87)
- KDAT: Inherent Adversarial Robustness via Knowledge Distillation with Adversarial Tuning for Object Detection Models (AAAI 2025 · similarity 0.85)
- Small Scale Data-Free Knowledge Distillation (CVPR 2024 · similarity 0.85)
- Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillation (AAAI 2024 · similarity 0.85)
- Neural Collapse Inspired Knowledge Distillation (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
