Research map / Architectures and efficiency / CNN design
REPrune: Channel Pruning via Kernel Representative Selection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes REPrune, a channel pruning method that clusters similar kernels within each input channel via Ward agglomerative clustering and greedily selects filters covering cluster representatives, integrated into a single concurrent training-pruning run.
Problem
Channel pruning removes whole filters, a coarse granularity that often causes accuracy drops; existing pipelines also often require pretrain-prune-finetune cycles.
Approach
Emulate kernel pruning with channel pruning: agglomerative (Ward) clustering of kernels within each input channel with a layer-wise linkage cut-off, then greedy maximum-cluster-coverage selection of filters that contain cluster representatives, applied progressively during training using BN scaling factors.
Key results
- On ImageNet ResNet-50, REPrune reaches 77.04% top-1 at 56.09% FLOPs reduction (+0.84% over baseline 76.2%), versus ACP and CUP at lower accuracy.
- On ImageNet ResNet-18, REPrune achieves 69.2% top-1 at 1.03G FLOPs, a 0.2% drop from the 69.4% baseline.
- On CIFAR-10 ResNet-56, REPrune gives 93.40% at 60.38% FLOPs reduction, versus 93.39% baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use REPrune rather than CUP at 43.09% FLOPs reduction | ResNet-18 · ImageNet | Top-1 accuracy: 67.37% -> 69.20% | Helped |
| Use REPrune rather than CUP at 56.09% FLOPs reduction | ResNet-50 · ImageNet | Top-1 accuracy: 74.34% -> 77.04% | Helped |
| Use REPrune rather than DMCP at 50% backbone FLOPs reduction | SSD300 with ResNet-50 backbone · COCO-2017 | mAP: 24.1 -> 25.0 | Helped |
| Replace Ward linkage with single linkage (ablation) | REPrune · CIFAR-10, ResNet-56 | Top-1 accuracy: 93.40% -> 92.74% | Hurt |
| Replace Ward linkage with complete linkage (ablation) | REPrune · CIFAR-10, ResNet-56 | Top-1 accuracy: 93.40% -> 92.67% | Hurt |
| Replace Ward linkage with average linkage (ablation) | REPrune · CIFAR-10, ResNet-56 | Top-1 accuracy: 93.40% -> 92.87% | Hurt |
| Set pruning end epoch T_prune to 75 rather than 180 (ablation) | REPrune · ImageNet, ResNet-18 | Top-1 accuracy: 69.20% -> 68.64% | Hurt |
| Set pruning end epoch T_prune to 125 rather than 180 (ablation) | REPrune · ImageNet, ResNet-18 | Top-1 accuracy: 69.20% -> 68.93% | Hurt |
| Set pruning end epoch T_prune to 250 rather than 180 (ablation) | REPrune · ImageNet, ResNet-18 | Top-1 accuracy: 69.20% -> 68.72% | Hurt |
| Convert concurrent training-pruning to train-prune-finetune (ablation) | REPrune · CIFAR-10, ResNet-56 | Top-1 accuracy: 93.40% -> 93.02% | Hurt |
| Replace random filter choice with max. ℓ2-norm selection (ablation) | REPrune · ImageNet, ResNet-18 | Top-1 accuracy: 69.48% -> 69.46% | No clear effect |
| Replace random filter choice with min. ℓ2-norm selection (ablation) | REPrune · ImageNet, ResNet-18 | Top-1 accuracy: 69.48% -> 69.57% | No clear effect |
Surprising findings
- Replacing Ward's linkage with single, complete, or average linkage limited accuracy retention for ResNet-56 at the same channel sparsity.
- Random seed and alternative selection criteria (max/min L2-norm) produced only small variation in ResNet-18 accuracy (69.44-69.57%).
Limitations
- Accuracy drops with alternative linkage methods suggest sensitivity to the clustering choice.
- Filter selection uses random sampling among tied candidates, so results vary with seed.
- Pruning applied only to the first conv layers in residual blocks due to residual connection constraints.
Experimental setup
Datasets: ImageNet CIFAR-10 COCO-2017 COCO-2014
Benchmarks: ImageNet CIFAR-10 COCO-2017 COCO-2014
Metrics: Top-1 accuracy FLOPs reduction mAP AP50 AP75 training time throughput
Baselines: SFP FPGM DMCP SOSP DTP CHEX CUP ACP LSC TMI-GKP AAP DMCP (SSD300)
Builds on: network slimming (BN scaling factor pruning) progressive channel pruning Ward's linkage TMI-GKP kernel pruning
Closest papers by meaning
- Rethinking the Smaller-Norm-Less-Informative Assumption in Channel Pruning of Convolution Layers (ICLR 2018 · similarity 0.90)
- CHEX: CHannel EXploration for CNN Model Compression (CVPR 2022 · similarity 0.89)
- Dynamic Channel Pruning: Feature Boosting and Suppression (ICLR 2019 · similarity 0.89)
- Storage Efficient and Dynamic Flexible Runtime Channel Pruning via Deep Reinforcement Learning (NeurIPS 2020 · similarity 0.89)
- RePr: Improved Training of Convolutional Filters (CVPR 2019 · similarity 0.89)
- BilevelPruning: Unified Dynamic and Static Channel Pruning for Convolutional Neural Networks (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.