Research map / Label-efficient and robust learning / Knowledge distillation
Hierarchical Knowledge Squeezed Adversarial Network Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| hierarchical connection of attention block instead of original (ablation) | HKSANC · CIFAR-10 | top-1 error (%): 7.55 -> 7.51 | No clear effect |
| hierarchical connection of attention block instead of original (ablation) | HKSANC · CIFAR-100 | top-1 error (%): 31.97 -> 31.86 | No clear effect |
| add intermediate loss Lis with squeezed knowledge to backbone loss (ablation) | HKSANC · CIFAR-10 | top-1 error (%): 8.19 -> 7.51 | Helped |
| add intermediate loss Lis with squeezed knowledge to backbone loss (ablation) | HKSANC · CIFAR-100 | top-1 error (%): 32.60 -> 31.86 | Helped |
| squeezed knowledge instead of attention map (AT) as transferred knowledge (ablation) | HKSANC · CIFAR-10 | top-1 error (%): 7.97 -> 7.51 | Helped |
| squeezed knowledge instead of VID, FSP or NST (ablation) | HKSANC · CIFAR-10 | top-1 error (%): 8.07 -> 7.51 | Helped |
| squeezed knowledge instead of AT on CIFAR-100 (ablation) | HKSANC · CIFAR-100 | top-1 error (%): 32.31 -> 31.86 | Helped |
| add category regularizer to adversarial loss on top of Lb+Lis (ablation) | HKSANC · CIFAR-10 | top-1 error (%): 7.51 -> 7.36 | Helped |
| add discriminator regularizer Lreg to adversarial loss on top of Lb+Lis (ablation) | HKSANC · CIFAR-10 | top-1 error (%): 7.51 -> 7.44 | Helped |
| use both category and discriminator regularizers (full adversarial loss) (ablation) | HKSANC · CIFAR-10 | top-1 error (%): 7.51 -> 7.29 | Helped |
| add adversarial loss Ladv to backbone loss only (ablation) | HKSANC · CIFAR-10 | top-1 error (%): 8.19 -> 7.72 | Helped |
| add Lis to Lb + Ladv (ablation) | HKSANC · CIFAR-10 | top-1 error (%): 7.72 -> 7.29 | Helped |
Closest papers by meaning
- Few Sample Knowledge Distillation for Efficient Network Compression (CVPR 2020 · similarity 0.90)
- Robust Knowledge Transfer via Hybrid Forward on the Teacher-Student Model (AAAI 2021 · similarity 0.88)
- Paraphrasing Complex Network: Network Compression via Factor Transfer (NeurIPS 2018 · similarity 0.87)
- N2N learning: Network to Network Compression via Policy Gradient Reinforcement Learning (ICLR 2018 · similarity 0.87)
- Knowledge distillation via softmax regression representation learning (ICLR 2021 · similarity 0.87)
- Hybrid Data-Free Knowledge Distillation (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.