Research map / Architectures and efficiency / CNN design
Faster CNNs with Direct Sparse Convolutions and Guided Pruning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use direct sparse convolution instead of lowering-based sparse convolution | AlexNet convolution · AlexNet conv5 | performance at high sparsity | Helped |
| Guide element-wise pruning toward layers with predicted speedup potential (ablation) | AlexNet · AlexNet conv2-5 | non-zero density: around 0.4 -> less than 0.2 | Helped |
| Guide dynamic network surgery toward layers with predicted speedup potential (ablation) | GoogLeNet · GoogLeNet layers with speedup potential | non-zero density reduction: 1.4× | Helped |
| Use element-wise sparsity instead of group-wise sparsity (ablation) | AlexNet larger convolution layers · AlexNet conv2-5 | non-zero density | Helped |
| Use structured sparsity learning rather than guided element-wise sparsity learning (ablation) | AlexNet · AlexNet convolution layers | achievable sparsity | Hurt |
| Avoid pruning 1×1 convolution layers and allocate pruning to 3×3 and 5×5 layers (ablation) | GoogLeNet GESL · GoogLeNet | accuracy | Helped |
| Prune 1×1 convolution layers with sparse convolution (ablation) | GoogLeNet · GoogLeNet 1×1 convolutions | inference speed | No clear effect |
| Prune layers with no predicted speedup potential (ablation) | Guided Sparsity Learning · CNN layers without predicted speedup potential | inference speed | Hurt |
| Increase sparsity beyond the upper bound of the useful range (ablation) | Sparse convolution · CNN convolution layers | inference speed | No clear effect |
| Use sparse-matrix sparse-matrix multiplication rather than sparse-matrix dense-matrix multiplication (ablation) | AlexNet fully connected layers · AlexNet fc layers | performance | Hurt |
Closest papers by meaning
- Pruning Filters for Efficient ConvNets (ICLR 2017 · similarity 0.92)
- Interspace Pruning: Using Adaptive Filter Representations To Improve Training of Sparse CNNs (CVPR 2022 · similarity 0.91)
- Fast Sparse ConvNets (CVPR 2020 · similarity 0.91)
- Inducing and Exploiting Activation Sparsity for Fast Inference on Deep Neural Networks (ICML 2020 · similarity 0.90)
- Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers (ICLR 2018 · similarity 0.90)
- PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-Time Execution on Mobile Devices (AAAI 2020 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.