Research map / Architectures and efficiency / Pruning and efficient tuning
Prior Gradient Mask Guided Pruning-Aware Fine-Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Guide fine-tuning with a prior gradient mask that scales updates to pruned filters | ResNet-20 · CIFAR-100 | accuracy: 66.85 -> 67.62 | Helped |
| Guide fine-tuning with a prior gradient mask that scales updates to pruned filters | ResNet-50 · ILSVRC-2012 | top-1 accuracy: 74.83 -> 75.11 | Helped |
| Use layer-wise pruning rates that increase from shallow to deep layers instead of the uniform-rate configuration | PGMPF on VGG16 · CIFAR-10 | accuracy: 93.46 -> 93.60 | Helped |
| Use the constant pruning rate strategy in SFP instead of the asymptotic pruning rate strategy (ablation) | PGMPF on VGG16 · CIFAR-100 | accuracy: 73.45 -> 73.66 | Helped |
| Add channel-wise random dropout of the prior gradient mask rather than using no dropout (ablation) | PGMPF on ResNet-20 · CIFAR-10 | accuracy: 92.34 -> 92.78 | Helped |
| Add channel-wise rather than layer-wise random dropout of the prior gradient mask (ablation) | PGMPF on ResNet-20 · CIFAR-10 | accuracy: 92.60 -> 92.78 | Helped |
| Add channel-wise random dropout of the prior gradient mask rather than using no dropout (ablation) | PGMPF on ResNet-56 at 20% pruning · CIFAR-10 | accuracy: 93.84 -> 95.07 | Helped |
| Add channel-wise rather than layer-wise random dropout of the prior gradient mask (ablation) | PGMPF on ResNet-56 at 20% pruning · CIFAR-10 | accuracy: 94.25 -> 95.07 | Helped |
| Add channel-wise rather than layer-wise random dropout of the prior gradient mask (ablation) | PGMPF on ResNet-56 at 40% pruning · CIFAR-10 | accuracy: 93.63 -> 94.36 | Helped |
| Set the initial weight-decay-mask rate α0 to 1 instead of 0 (ablation) | PGMPF on ResNet-20 at 20% pruning · CIFAR-10 | accuracy: 92.56 -> 92.78 | Helped |
| Set the initial weight-decay-mask rate α0 to 1 instead of 0 (ablation) | PGMPF on ResNet-20 at 40% pruning · CIFAR-10 | accuracy: 91.49 -> 91.54 | Helped |
| Set the initial weight-decay-mask rate α0 to 1 instead of 0 (ablation) | PGMPF on ResNet-56 at 20% pruning · CIFAR-10 | accuracy: 94.97 -> 95.07 | Helped |
Closest papers by meaning
- Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks (NeurIPS 2021 · similarity 0.89)
- Gate Decorator: Global Filter Pruning Method for Accelerating Deep Convolutional Neural Networks (NeurIPS 2019 · similarity 0.89)
- Structured Pruning of Neural Networks With Budget-Aware Regularization (CVPR 2019 · similarity 0.88)
- Learning Pruning-Friendly Networks via Frank-Wolfe: One-Shot, Any-Sparsity, And No Retraining (ICLR 2022 · similarity 0.88)
- Pruning Filters for Efficient ConvNets (ICLR 2017 · similarity 0.88)
- Learning Filter Pruning Criteria for Deep Convolutional Neural Networks Acceleration (CVPR 2020 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.