Research map / Architectures and efficiency / Pruning and efficient tuning
Model Preserving Compression for Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ID pruning rather than magnitude pruning | VGG-16 · CIFAR-10 | correlation: 93.15 -> 97.29 | Helped |
| Compose ID with LRank rather than use ID alone | VGG-16 · CIFAR-10 | accuracy: 93.31 -> 93.43 | No clear effect |
| Compose ID with LRank rather than use ID alone | VGG-16 · CIFAR-10 | correlation: 97.29 -> 95.9 | Hurt |
| Compose ID with LRank and fine-tune for 10 epochs rather than use the zero-epoch composition | VGG-16 · ImageNet | accuracy: 68.84 -> 70.50 | Helped |
| Compose ID with LRank and fine-tune for 10 epochs rather than use the zero-epoch composition | VGG-16 · ImageNet | correlation: 85.66 -> 90.90 | Helped |
| Prune with ID and fine-tune rather than train from scratch | MobileNet V1 · CIFAR-10 | correlation: 88.6% -> 92.6% | Helped |
| Use ID pruning rather than magnitude pruning | One-hidden-layer network · Synthetic 2D regression | test loss: 0.33 -> 0.037781 | Helped |
| Remove the class from the pruning set while retaining it in train and test data (ablation) | ID-pruned VGG-16 · CIFAR-10, omitted class | sub-class accuracy | Hurt |
| Remove the class from the pruning set and use magnitude pruning instead of ID (ablation) | VGG-16 · CIFAR-10, omitted class | sub-class accuracy | Hurt |
| Use a different training data order for magnitude-pruning fine-tuning (ablation) | VGG-16 pruned to 50% FLOPs · CIFAR-10 | correlation: 94.01% -> 93.15% | Hurt |
| Use a larger learning rate during magnitude-pruning fine-tuning (ablation) | VGG-16 pruned to 50% FLOPs · CIFAR-10 | correlation | Hurt |
| Use a smaller pruning fraction per iteration (ablation) | Iterative ID · CIFAR-10, VGG-16 | accuracy | No clear effect |
Closest papers by meaning
- Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition (NeurIPS 2021 · similarity 0.92)
- Compression-aware Training of Deep Networks (NeurIPS 2017 · similarity 0.91)
- Distribution-Aware Tensor Decomposition for Compression of Convolutional Neural Networks (NeurIPS 2025 · similarity 0.89)
- Forget the Data and Fine-Tuning! Just Fold the Network to Compress (ICLR 2025 · similarity 0.88)
- Auto-Compressing Networks (NeurIPS 2025 · similarity 0.88)
- Layer Compression of Deep Networks with Straight Flows (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.