Research map / Architectures and efficiency / CNN design
Fast Sparse ConvNets
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace dense convolutions with sparse convolutions and use sparse inference kernels | MobileNet v1 · ImageNet validation, 224x224 | SD835 inference time: 125 -> 58 | Helped |
| Replace dense convolutions with sparse convolutions and use sparse inference kernels | MobileNet v2 · ImageNet validation, 224x224 | SD835 inference time: 150 -> 93 | Helped |
| Replace dense convolutions with sparse convolutions and use sparse inference kernels | EfficientNet · ImageNet validation, 224x224 | SD835 inference time: 158 -> 110 | Helped |
| Use 80% rather than the main model's 85% sparsity, with block size 1 throughout (ablation) | MobileNet v2 Sparse · ImageNet validation, 224x224 | Top-1 accuracy: 74.5 -> 74.9 | Helped |
| Use 80% rather than the main model's 85% sparsity, with block size 1 throughout (ablation) | MobileNet v2 Sparse · ImageNet validation, 224x224 | SD835 inference time: 93 -> 102 | Hurt |
| Use the cache-aware MBv2 architecture, varying expansion factor to fit cache and adding layers (ablation) | MobileNet v2 Sparse · ImageNet validation, 224x224 | SD835 inference time: 40 -> 33 | Helped |
| Use block sparsity instead of unstructured sparsity (ablation) | MobileNet v1 and MobileNet v2 · ImageNet | Top-1 accuracy | Hurt |
| Increase sparsity from 70% to 90% (ablation) | MobileNet v1 and MobileNet v2 · ImageNet | FLOPs at a given Top-1 accuracy | Helped |
| Increase sparsity above 80% (ablation) | EfficientNet · ImageNet | Efficiency | No clear effect |
| Extend training and slow learning-rate decay as done for sparse EfficientNet (ablation) | Dense EfficientNet baseline · ImageNet | Top-1 accuracy | No clear effect |
| Extend training by a factor of four for sparsification (ablation) | Sparse MobileNet v1 and MobileNet v2 · ImageNet | Top-1 accuracy | Helped |
| Prune the final fully connected layer (ablation) | MobileNet v1 and MobileNet v2 · ImageNet | Top-1 accuracy | Hurt |
Closest papers by meaning
- Faster CNNs with Direct Sparse Convolutions and Guided Pruning (ICLR 2017 · similarity 0.91)
- Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together! (ICLR 2023 · similarity 0.91)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.90)
- Sparse maximal update parameterization: A holistic approach to sparse training dynamics (NeurIPS 2024 · similarity 0.88)
- Sparse Cocktail: Every Sparse Pattern Every Sparse Ratio All At Once (ICML 2024 · similarity 0.88)
- Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.