Research map / Architectures and efficiency / Pruning and efficient tuning
Ensemble Knowledge Guided Sub-network Search and Fine-Tuning for Filter Pruning
Seunghyun Lee, Byung Cheol Song
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add single (pre-trained) teacher knowledge to the search reward (ablation) | EKG search, ResNet-56 · CIFAR10 | accuracy (%): 93.78 -> 93.54 | Hurt |
| add single (pre-trained) teacher knowledge to the search reward (ablation) | EKG search, ResNet-56 · CIFAR100 | accuracy (%): 71.63 -> 71.66 | No clear effect |
| add single teacher to search reward (ablation) | EKG search, MobileNet-v2 · CIFAR100 | accuracy (%): 74.27 -> 74.10 | Hurt |
| use ensemble of interim sub-networks instead of single teacher in the search reward (ablation) | EKG search, ResNet-56 · CIFAR10 | accuracy (%): 93.54 -> 93.85 | Improved |
| add ensemble knowledge to the search reward (vs none) (ablation) | EKG search, MobileNet-v2 · CIFAR10 | accuracy (%): 93.69 -> 93.89 | Improved |
| add single-teacher fine-tuning guidance after ensemble search (ablation) | EKG, ResNet-56 · CIFAR100 | accuracy (%): 71.82 -> 72.62 | Improved |
| use ensemble memory bank instead of single teacher in fine-tuning (ablation) | EKG, ResNet-56 · CIFAR100 | accuracy (%): 72.62 -> 72.93 | Improved |
| use ensemble memory bank instead of single teacher in fine-tuning (ablation) | EKG, MobileNet-v2 · CIFAR100 | accuracy (%): 76.11 -> 76.29 | Improved |
| use ensemble memory bank instead of single teacher in fine-tuning (ablation) | EKG, ResNet-56 · fine-tuning training time | training time increase: 71.09% -> 29.03% training time increase | Improved |
| use EKG score (condition-number-aware search) vs GBN score | EKG search, ResNet-56 · CIFAR10 | accuracy (%): 93.70 -> 93.85 | Improved |
| EKG vs GBN despite higher validation loss | EKG search, ResNet-56 · CIFAR100 | accuracy (%): 0.6816 -> 0.7733 validation loss; 71.60 -> 71.82 test accuracy | Improved |
| plug EKG into Autoslim greedy search | Autoslim, ResNet-50 · ImageNet-2012 | top-1 accuracy (%): +0.34% top-1, +3.5% FLOPs reduction | Improved |
Closest papers by meaning
- Adaptive Search-and-Training for Robust and Efficient Network Pruning (TPAMI 2023 · similarity 0.88)
- Rethinking the Value of Network Pruning (ICLR 2019 · similarity 0.87)
- Connectivity Matters: Neural Network Pruning Through the Lens of Effective Sparsity (JMLR 2023 · similarity 0.86)
- SCOP: Scientific Control for Reliable Neural Network Pruning (NeurIPS 2020 · similarity 0.86)
- Network Pruning via Transformable Architecture Search (NeurIPS 2019 · similarity 0.86)
- Prune and Tune Ensembles: Low-Cost Ensemble Learning with Sparse Independent Subnetworks (AAAI 2022 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
