Research map / Architectures and efficiency / Neural architecture search
Network Pruning via Transformable Architecture Search
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply KD to the searched network rather than using no knowledge transfer (ablation) | TAS† on ResNet-32 · CIFAR-100, about 40% FLOPs pruned | accuracy: 68.95 % -> 72.41 % | Helped |
| Initialize the searched network from the pretrained unpruned network (ablation) | TAS† on ResNet-32 · CIFAR-100, about 40% FLOPs pruned | accuracy: 68.95 % -> 69.70 % | Helped |
| Search both width and depth instead of depth only (ablation) | TAS on ResNet-32 · CIFAR-10 | accuracy: 91.48% -> 93.16% | Helped |
| Search both width and depth instead of width only (ablation) | TAS on ResNet-32 · CIFAR-100 | accuracy: 71.74% -> 72.41% | Helped |
| Use channel-wise interpolation with Gumbel-softmax sampling rather than omit interpolation or use vanilla softmax (ablation) | TAS architecture search · CIFAR-100, ResNet-56, unconstrained FLOPs | architecture selection confidence | Helped |
| Aggregate more than one architecture sample instead of one (ablation) | TAS on ResNet-32 · CIFAR-10 | accuracy: 89.85% -> 92.98% | Helped |
| Vary sample count among |I|>1 (ablation) | TAS on ResNet-32 · CIFAR-10 | accuracy: 92.98% (|I|=2); 92.63% (|I|=3); 93.18% (|I|=5); 92.65% (|I|=8) | No clear effect |
| Use TAS rather than FPGM | ResNet-56 · CIFAR-100 | accuracy: 69.66% -> 72.25% | Helped |
| Use TAS rather than AutoSlim | ResNet-50 · ImageNet | Top-1 accuracy: 76.00% -> 76.20% | Helped |
| Use TAS rather than LCCL | ResNet-164 · CIFAR-10 | pruned accuracy: 94.09% -> 94.00% | Hurt |
Closest papers by meaning
- Pruning from Scratch (AAAI 2020 · similarity 0.92)
- Rethinking the Value of Network Pruning (ICLR 2019 · similarity 0.89)
- A Probabilistic Approach to Neural Network Pruning (ICML 2021 · similarity 0.89)
- Convolutional Neural Network Pruning With Structural Redundancy Reduction (CVPR 2021 · similarity 0.89)
- How Sparse Can We Prune A Deep Network: A Fundamental Limit Perspective (NeurIPS 2024 · similarity 0.87)
- Device-Wise Federated Network Pruning (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.