Research map / Architectures and efficiency / Pruning and efficient tuning
Neural Network Pruning With Residual-Connections and Limited-Data
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Prune channels inside and outside residual connections using a KL-divergence criterion, producing a wallet-shaped structure | ResNet50 · ImageNet | Top-1 Acc.: 73.39% vs. 73.05% | Helped |
| Use wallet-shaped pruning instead of hourglass-shaped pruning | ResNet50 · ImageNet, 1.39G MACs | inference time per mini-batch of 256 images: 0.21s -> 0.19s | Helped |
| Directly prune MobileNetV2-1.0 on the target dataset with CURL instead of fine-tuning MobileNetV2-0.5 | MobileNetV2 · CUB200-2011 | accuracy: 73.96% -> 78.72% | Helped |
| Add mixup to standard fine-tuning (ablation) | CURL · CUB200-2011 | accuracy: 70.76 -> 73.89 | Helped |
| Add knowledge distillation to standard fine-tuning (ablation) | CURL · CUB200-2011 | accuracy: 70.76 -> 77.87 | Helped |
| Combine mixup with knowledge distillation (ablation) | CURL · CUB200-2011 | accuracy: 77.87 -> 77.99 | Helped |
| Add data expansion and label refinement after mixup plus knowledge distillation (ablation) | CURL · CUB200-2011 | accuracy: 77.99 -> 78.72 | Helped |
| Train the pruned model from scratch instead of fine-tuning pretrained weights (ablation) | CURL · CUB200-2011 | accuracy: 70.76 -> 64.39 | Hurt |
| Use KL-divergence rather than random selection, weight sum, or loss change to score filters (ablation) | MobileNetV2 · CUB200-2011, pruning without fine-tuning | accuracy: 6.80% / 8.25% / 21.83% -> 66.33% | Helped |
| Add label refinement (ablation) | MobileNetV2 · CUB200-2011 | accuracy: 77.43% -> 78.72% | Helped |
| Add label refinement (ablation) | ResNet50 · Oxford Pets | accuracy: 90.32% -> 90.30% | No clear effect |
| Update teacher soft targets during label refinement rather than keeping them fixed (ablation) | MobileNetV2 · CUB200-2011 | accuracy: 78.56% -> 78.72% | Helped |
Closest papers by meaning
- Structured Pruning of Neural Networks With Budget-Aware Regularization (CVPR 2019 · similarity 0.91)
- Pruning from Scratch (AAAI 2020 · similarity 0.90)
- Pruning Filters for Efficient ConvNets (ICLR 2017 · similarity 0.90)
- Pruning Convolutional Neural Networks for Resource Efficient Inference (ICLR 2017 · similarity 0.90)
- Distilling the Knowledge in Data Pruning (ICML 2025 · similarity 0.89)
- BilevelPruning: Unified Dynamic and Static Channel Pruning for Convolutional Neural Networks (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.