Research map / Architectures and efficiency / Pruning and efficient tuning
Deep Rewiring: Training very sparse deep networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Constrain connectivity to 1.3% | DEEP R feed-forward network · MNIST | accuracy: 96.2 % | Helped |
| Constrain connectivity to 1.0% and double the training epochs | DEEP R feed-forward network · MNIST | accuracy: 96.3% | Helped |
| Constrain connectivity to 5% | DEEP R CNN · CIFAR-10 | accuracy: 84.1 % | Helped |
| Use DEEP R instead of soft-DEEP R at low maximum connectivity (ablation) | Sparse networks · MNIST and CIFAR-10 | accuracy | Helped |
| Use DEEP R instead of fixed random connectivity (ablation) | Sparse neural networks · TIMIT | phoneme error rate | Helped |
| Use DEEP R or soft-DEEP R instead of pruning or ℓ1-shrinkage (ablation) | Sparse neural networks · MNIST and TIMIT at very low connectivity | performance | Helped |
| Use pruning instead of DEEP R (ablation) | Sparse neural networks · MNIST at larger connectivities | accuracy | Helped |
| Use DEEP R or soft-DEEP R instead of pruning or ℓ1-shrinkage (ablation) | Recurrent LSTM · TIMIT | phoneme error rate stability | Helped |
| Keep temperature constant so rewiring continues during training | DEEP R · Class-shuffled MNIST | classification accuracy across task shuffles | Helped |
| Vary temperature (ablation) | DEEP R · MNIST, CIFAR-10, and TIMIT | performance | No clear effect |
| Choose an unsuitable ℓ1 regularization coefficient (ablation) | DEEP R · MNIST, CIFAR-10, and TIMIT | accuracy | Hurt |
Closest papers by meaning
- Training Adversarially Robust Sparse Networks via Bayesian Connectivity Sampling (ICML 2021 · similarity 0.88)
- Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining (ICLR 2022 · similarity 0.87)
- One-Shot Pruning of Recurrent Neural Networks by Jacobian Spectrum Evaluation (ICLR 2020 · similarity 0.86)
- Locality-Aware Graph Rewiring in GNNs (ICLR 2024 · similarity 0.85)
- Global Minimizers of $\ell^p$-Regularized Objectives Yield the Sparsest ReLU Neural Networks (NeurIPS 2025 · similarity 0.85)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.