Research map / Architectures and efficiency / Pruning and efficient tuning
DropNet: Reducing Neural Network Complexity via Iterative Pruning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use minimum (global) post-activation pruning metric instead of maximum/random | Model A: FC40 - FC40 MLP · MNIST | test accuracy vs fraction of nodes remaining | Helped |
| Use layer-wise minimum post-activation metric (minimum layer) | Model B: Conv64 - Conv64 CNN · MNIST | test accuracy vs fraction of filters remaining | Helped |
| Use layer-wise minimum post-activation metric (minimum layer) | Model C: Conv64 - Conv64 - Conv128 - Conv128 · CIFAR-10 | test accuracy vs fraction of filters remaining | Helped |
| Use layer-wise minimum post-activation metric (minimum layer) | ResNet18 and VGG19 · CIFAR-10 | test accuracy vs fraction of filters remaining | Helped |
| Use global minimum metric instead of global maximum metric | Model A, B, C, ResNet18, VGG19 · MNIST and CIFAR-10 | test accuracy vs fraction of nodes/filters remaining | Helped |
| Use global minimum metric for pruning on Model A FC20-FC20 compared against oracle (ablation) | Model A: FC20 - FC20 · MNIST oracle comparison | test accuracy vs fraction of nodes remaining | No clear effect |
| Increase pruning fraction p per cycle from 0.2 to 0.9 (ablation) | Model C (Conv64-Conv64-Conv128-Conv128) minimum metric · CIFAR-10 | test accuracy vs fraction of filters remaining | Hurt |
| Increase pruning fraction p from 0.2 to 0.3-0.5 per cycle (ablation) | minimum metric DropNet · MNIST and CIFAR-10 | test accuracy vs fraction remaining | No clear effect |
| Random re-initialization of pruned architecture instead of original initialization (ablation) | DropNet (Model A FC20-FC20, Model B, Model C) · MNIST and CIFAR-10 | test accuracy vs fraction remaining | No clear effect |
Closest papers by meaning
- Net-Trim: Convex Pruning of Deep Neural Networks with Performance Guarantee (NeurIPS 2017 · similarity 0.89)
- Structured Pruning of Neural Networks With Budget-Aware Regularization (CVPR 2019 · similarity 0.89)
- Pruning neural networks without any data by iteratively conserving synaptic flow (NeurIPS 2020 · similarity 0.89)
- Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeon (NeurIPS 2017 · similarity 0.89)
- Gradual DropIn of Layers to Train Very Deep Neural Networks (CVPR 2016 · similarity 0.88)
- Neural Network Pruning Denoises the Features and Makes Local Connectivity Emerge in Visual Tasks (ICML 2022 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.