Research map / Architectures and efficiency / Pruning and efficient tuning
ThiNet: A Filter Level Pruning Method for Deep Neural Network Compression
Jian-Hao Luo, Jianxin Wu, Weiyao Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add least-squares channel weighting to ThiNet filter selection (ablation) | ThiNet on VGG-16-GAP · CUB-200, conv1-1 60% filters removed, before fine-tuning | top-1 accuracy (%): 68.24% -> 70.75% | Improved |
| select filters with ThiNet instead of weight sum | VGG-16-GAP pruning · CUB-200, conv1-1 60% filters removed, before fine-tuning | top-1 accuracy (%): 40.99% -> 68.24% | Improved |
| select filters with ThiNet instead of random selection | VGG-16-GAP pruning · CUB-200, conv1-1 60% filters removed, before fine-tuning | top-1 accuracy (%): 51.26% -> 68.24% | Improved |
| prune pre-trained VGG-16 and fine-tune instead of training the same structure from scratch (ablation) | ThiNet-Conv · ImageNet val (VGG-16) | top-1 accuracy (%): 67.00% -> 69.80% | Improved |
| replace FC layers with global average pooling (ThiNet-GAP) | ThiNet-Conv VGG-16 · ImageNet val (VGG-16) | top-1 accuracy (%): 69.80% -> 67.34% | Hurt |
| select filters with ThiNet instead of weight sum (ThiNet-WS) in the full framework | ThiNet-Conv VGG-16 · ImageNet val (VGG-16) | top-1 accuracy change (%): +1.01% -> +1.46% | Improved |
| increase compression to preserve only 30% of filters instead of 70% (ablation) | ThiNet on ResNet-50 · ImageNet val (ResNet-50) | top-1 accuracy (%): 72.04% -> 68.42% | Hurt |
| prune on ImageNet first then fine-tune on target (ThiNet-GAP) instead of fine-tune then prune | VGG-16 on CUB-200 · CUB-200 (domain-specific small model) | top-1 accuracy (%): 66.90% -> 69.43% | Improved |
| prune on ImageNet first then fine-tune on target (ThiNet-GAP) instead of fine-tune then prune | VGG-16 on Indoor-67 · Indoor-67 (domain-specific small model) | top-1 accuracy (%): 64.70% -> 70.22% | Improved |
| use ThiNet-GAP pruned model instead of training same structure from scratch | VGG-16 on CUB-200 · CUB-200 (domain-specific small model) | top-1 accuracy (%): 44.27% -> 69.43% | Improved |
| use ThiNet-Tiny instead of AlexNet when transferred to domain-specific tasks | small model on CUB-200 · CUB-200 (domain-specific small model) | top-1 accuracy (%): 57.28% -> 65.45% | Improved |
| use ThiNet-Tiny instead of SqueezeNet | compact VGG-16 model · ImageNet val (VGG-16) | top-1 accuracy (%): 57.67% -> 59.34% | Improved |
Later papers that compared with this method
22 later accepted papers report ThiNet as a baseline in their results tables; 19 of them report a better number on most of the results they share with it.
- OSSCAR: One-Shot Structured Pruning in Vision and Language Models with Combinatorial Optimization (ICML 2024 · better on 2 of 2 shared results)
- Balanced Column-Wise Block Pruning for Maximizing GPU Parallelism (AAAI 2023 · better on 1 of 1 shared result)
- Performance-Aware Approximation of Global Channel Pruning for Multitask CNNs (TPAMI 2023 · better on 1 of 1 shared result)
- Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equations (NeurIPS 2022 · better on 1 of 2 shared results)
- Convolutional Neural Network Compression through Generalized Kronecker Product Decomposition (AAAI 2022 · better on 1 of 1 shared result)
- Compressing Models With Few Samples: Mimicking Then Replacing (CVPR 2022 · better on 1 of 1 shared result)
- Discrimination-aware Network Pruning for Deep Model Compression (TPAMI 2021 · better on 3 of 3 shared results)
- Towards Compact CNNs via Collaborative Compression (CVPR 2021 · better on 1 of 1 shared result)
14 more not listed.
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- ThiNet: Pruning CNN Filters for a Thinner Net (TPAMI 2018 · similarity 0.95)
- Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks (NeurIPS 2019 · similarity 0.91)
- Coreset-Based Neural Network Compression (ECCV 2018 · similarity 0.91)
- Pruning Filters for Efficient ConvNets (ICLR 2017 · similarity 0.91)
- Auto-Balanced Filter Pruning for Efficient Convolutional Neural Networks (AAAI 2018 · similarity 0.91)
- LayerMerge: Neural Network Depth Compression through Layer Pruning and Merging (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
