Research map / Image generation / GANs
Data-Free Learning of Student Networks
Hanting Chen, Yunhe Wang, Chang Xu, Zhaohui Yang, Chuanjian Liu, Boxin Shi, Chunjing Xu, Chao Xu, Qi Tian
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add information entropy loss to one-hot + feature activation losses (ablation) | DAFL generator · MNIST (LeNet-5 to LeNet-5-HALF) | top-1 accuracy: 42.07% -> 98.20% | Improved |
| add feature maps activation loss to one-hot + information entropy losses (ablation) | DAFL generator · MNIST (LeNet-5 to LeNet-5-HALF) | top-1 accuracy: 97.25% -> 98.20% | Improved |
| add one-hot loss to information entropy + feature activation losses (ablation) | DAFL generator · MNIST (LeNet-5 to LeNet-5-HALF) | top-1 accuracy: 95.53% -> 98.20% | Improved |
| train generator with one-hot loss only instead of an untrained generator (ablation) | DAFL generator · MNIST (LeNet-5 to LeNet-5-HALF) | top-1 accuracy: 88.01% -> 78.77% | Hurt |
| train generator with feature maps activation loss only instead of an untrained generator (ablation) | DAFL generator · MNIST (LeNet-5 to LeNet-5-HALF) | top-1 accuracy: 88.01% -> 15.95% | Hurt |
| train generator with information entropy loss only instead of an untrained generator (ablation) | DAFL generator · MNIST (LeNet-5 to LeNet-5-HALF) | top-1 accuracy: 88.01% -> 88.14% | No clear effect |
| replace normal-distribution noise with the DAFL generated images | data-free KD of student · MNIST (LeNet-5) | accuracy: 88.01% -> 98.20% | Improved |
| replace USPS alternative data with DAFL generated images | data-free KD of student · MNIST (LeNet-5) | accuracy: 94.56% -> 98.20% | Improved |
| replace meta-data reconstruction (Lopes et al.) with DAFL | data-free KD of student · MNIST (LeNet-5) | accuracy: 92.47% -> 98.20% | Improved |
| replace CIFAR-100 alternative data with DAFL generated images | data-free KD of student · CIFAR-10 (ResNet-34 to ResNet-18) | accuracy: 90.65% -> 92.22% | Improved |
| replace CIFAR-10 alternative data with DAFL generated images | data-free KD of student · CIFAR-100 (ResNet-34 to ResNet-18) | accuracy: 69.88% -> 74.47% | Improved |
| replace meta-data reconstruction with DAFL | data-free KD of student · CelebA (AlexNet to AlexNet-Half) | accuracy: 77.56% -> 80.03% | Improved |
Closest papers by meaning
- Adversarial Learning of Portable Student Networks (AAAI 2018 · similarity 0.93)
- Learning Student Networks in the Wild (CVPR 2021 · similarity 0.89)
- Hybrid Data-Free Knowledge Distillation (AAAI 2025 · similarity 0.89)
- Distilling Portable Generative Adversarial Networks for Image Translation (AAAI 2020 · similarity 0.88)
- Teachers Do More Than Teach: Compressing Image-to-Image Models (CVPR 2021 · similarity 0.87)
- Small Scale Data-Free Knowledge Distillation (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
