Research map / Architectures and efficiency / Pruning and efficient tuning
Interspace Pruning: Using Adaptive Filter Representations To Improve Training of Sparse CNNs
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace spatial-weight pruning with adaptive filter-basis coefficient pruning | ResNet50 with FT · ImageNet val, p=0.9 | top-1 accuracy: 75.67 ± 0.09 -> 75.89 ± 0.09 | Helped |
| Replace spatial-weight pruning with adaptive filter-basis coefficient pruning | ResNet50 with GMP · ImageNet val, p=0.9 | top-1 accuracy: 73.57 ± 0.06 -> 74.20 ± 0.07 | Helped |
| Replace spatial-weight pruning with adaptive filter-basis coefficient pruning | ResNet50 with RigL · ImageNet val, p=0.9 | top-1 accuracy: 73.88 ± 0.06 -> 74.32 ± 0.11 | Helped |
| Replace spatial-weight pruning with adaptive filter-basis coefficient pruning | VGG16 with SET · CIFAR-10 test, p=0.99 | accuracy: 89.36 -> 90.92 | Helped |
| Replace spatial-weight pruning with adaptive filter-basis coefficient pruning | VGG16 with SNIP · CIFAR-10 test, p=0.99 | accuracy: 87.75 -> 90.79 | Helped |
| Use coarse rather than fine filter-basis sharing (ablation) | VGG16 with SNIP · CIFAR-10 test, p=0.85 | accuracy: 93.2 ± 0.1 -> 93.5± 0.0 | Helped |
| Use coarse rather than fine filter-basis sharing (ablation) | VGG16 with SET · CIFAR-10 test, p=0.85 | accuracy: 93.6 ± 0.2 -> 93.7 ± 0.2 | Helped |
| Use fine rather than coarse filter-basis sharing (ablation) | VGG16 with IP-SNIP · CIFAR-10 test, p=0.85 | accuracy: 93.5± 0.0 -> 93.2 ± 0.1 | Hurt |
| Use random ONB initialization instead of standard initialization (ablation) | VGG16 with IP-SNIP · CIFAR-10 | test accuracy | Mixed |
| Use non-orthonormal random filter bases rather than random ONBs (ablation) | VGG16 with IP-SNIP · CIFAR-10 | test accuracy | Mixed |
| Apply weight decay to FBs as well as FB coefficients instead of coefficients alone (ablation) | VGG16 with IP-SNIP · CIFAR-10, lower pruning rates | test accuracy | Helped |
| Apply weight decay to FBs (ablation) | VGG16 with IP-SNIP · CIFAR-10, higher pruning rates | test accuracy | Hurt |
Closest papers by meaning
- Pruning Filters for Efficient ConvNets (ICLR 2017 · similarity 0.92)
- Faster CNNs with Direct Sparse Convolutions and Guided Pruning (ICLR 2017 · similarity 0.91)
- Provable Filter Pruning for Efficient Neural Networks (ICLR 2020 · similarity 0.91)
- Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration (CVPR 2020 · similarity 0.91)
- RePr: Improved Training of Convolutional Filters (CVPR 2019 · similarity 0.91)
- Entropy Induced Pruning Framework for Convolutional Neural Networks (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.