Research map / Architectures and efficiency / Pruning and efficient tuning
Parameter efficient training of deep convolutional neural networks by dynamic sparse reparameterization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use dynamic sparse reallocation instead of compressed sparse training | ResNet-50 · ImageNet, sparsity 0.8 | top-1 accuracy (%): 73.2 -> 73.3 | Helped |
| Use dynamic sparse reallocation instead of SET | ResNet-50 · ImageNet, sparsity 0.8 | top-1 accuracy (%): 72.6 -> 73.3 | Helped |
| Use dynamic sparse reallocation instead of DeepR | ResNet-50 · ImageNet, sparsity 0.8 | top-1 accuracy (%): 71.7 -> 73.3 | Helped |
| Use dynamic sparse reallocation instead of static sparse training | ResNet-50 · ImageNet, sparsity 0.8 | top-1 accuracy (%): 71.6 -> 73.3 | Helped |
| Use dynamic sparse reallocation instead of HashedNet parameter sharing | ResNet-50 · ImageNet, sparsity 0.8 | top-1 accuracy (%): 70.0 -> 73.3 | Helped |
| Fix the final sparse connectivity and randomly reinitialize weights rather than continue dynamic reallocation (ablation) | Dynamic sparse WRN-28-2 · CIFAR10 | test accuracy | Hurt |
| Fix the final sparse connectivity and use the original initialization rather than continue dynamic reallocation (ablation) | Dynamic sparse WRN-28-2 · CIFAR10 | test accuracy | Hurt |
| Stop dynamic reallocation after initial training epochs and keep optimizing weights with fixed structure (ablation) | Dynamic sparse WRN-28-2 · CIFAR10 | test accuracy | No clear effect |
| Constrain sparsity to 3×3 kernel slices instead of unstructured weights (ablation) | Dynamic sparse CNN · Not specified | generalization performance | Hurt |
Closest papers by meaning
- Dynamic Sparse Training: Find Efficient Sparse Network From Scratch With Trainable Masked Layers (ICLR 2020 · similarity 0.89)
- Reparameterization through Spatial Gradient Scaling (ICLR 2023 · similarity 0.89)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.88)
- Sparse maximal update parameterization: A holistic approach to sparse training dynamics (NeurIPS 2024 · similarity 0.88)
- Global Minimizers of $\ell^p$-Regularized Objectives Yield the Sparsest ReLU Neural Networks (NeurIPS 2025 · similarity 0.87)
- Dynamic Sparse Training versus Dense Training: The Unexpected Winner in Image Corruption Robustness (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.