Research map / Architectures and efficiency / Pruning and efficient tuning
Influence Function Based Second-Order Channel Pruning: Evaluating True Loss Changes for Pruning is Possible Without Retraining
Hongrong Cheng, Miao Zhang, Qinfeng Shi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace GFP criterion with IFSO influence-function second-order criterion | GFP · CIFAR-10 VGG-16 (40% FLOPs) | Top-1 accuracy: 93.72 -> 94.04 | Improved |
| replace GFP criterion with IFSO | GFP · CIFAR-10 ResNet-32 (50% FLOPs) | Top-1 accuracy: 92.64 -> 93.00 | Improved |
| replace GFP criterion with IFSO | GFP · CIFAR-100 ResNet-56 (50% FLOPs) | Top-1 accuracy: 70.17 -> 70.55 | Improved |
| replace GFP criterion with IFSO | GFP · COCO 2017 RetinaNet (60% FLOPs) | mAP: 35.30 -> 35.70 | Improved |
| use IFSO on ImageNet ResNet-50 at 40% FLOPs vs SFP | SFP · ImageNet ResNet-50 (40% FLOPs) | Top-1 accuracy: 74.61 -> 75.70 | Improved |
| replace layer-wise normalization of channel scores with IFSO scoring (ablation) | Norm-40 · CIFAR-10 ResNet-32 (40% FLOPs) | Top-1 accuracy: 93.11 -> 93.23 | Improved |
| replace raw scores with IFSO scoring (ablation) | Raw-40 · CIFAR-10 ResNet-32 (40% FLOPs) | Top-1 accuracy: 92.83 -> 93.23 | Improved |
| normalization policies on ResNet-56 at 50% FLOPs (ablation) | Raw-50 · CIFAR-10 ResNet-56 (50% FLOPs) | Top-1 accuracy: 93.65 -> 93.72 | Mixed |
| use identity instead of Neumann-1 approximation of inverse Hessian (ablation) | Neumann-1-40 · CIFAR-10 ResNet-32 (40% FLOPs) | Top-1 accuracy: 93.14 -> 93.23 | Improved |
| use identity instead of Neumann-2 approximation (ablation) | Neumann-2-50 · CIFAR-10 ResNet-32 (50% FLOPs) | Top-1 accuracy: 92.67 -> 93.00 | Improved |
| accumulate channel scores k times instead of using the k-th scores (ablation) | IFSO · CIFAR-10 ResNet-32 (50% FLOPs) | Top-1 accuracy | Improved |
| use more batches of proxy dataset (ablation) | IFSO · CIFAR-10 | Top-1 accuracy | No clear effect |
Closest papers by meaning
- UPSCALE: Unconstrained Channel Pruning (ICML 2023 · similarity 0.90)
- Network Pruning via Performance Maximization (CVPR 2021 · similarity 0.90)
- Revisiting Random Channel Pruning for Neural Network Compression (CVPR 2022 · similarity 0.90)
- Discrimination-aware Channel Pruning for Deep Neural Networks (NeurIPS 2018 · similarity 0.90)
- Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers (ICLR 2018 · similarity 0.90)
- EPSD: Early Pruning with Self-Distillation for Efficient Model Compression (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
