Research map / Architectures and efficiency / Pruning and efficient tuning
Learning Best Combination for Efficient N:M Sparsity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Learn combination scores jointly with weights and progressively prune low-scored combinations | ResNet-50, 2:4 N:M sparsity · ImageNet | Top-1 accuracy (%): 77.0 -> 77.2 | Helped |
| Use learned combination scores instead of magnitude-based selection | ResNet-50, 2:4 N:M sparsity · ImageNet | Top-1 accuracy (%): 75.8 -> 77.2 | Helped |
| Use learned combination scores instead of gradient-based selection | ResNet-50, 2:4 N:M sparsity · ImageNet | Top-1 accuracy (%): 76.1 -> 77.2 | Helped |
| Reverse the ordering of learned combination scores (ablation) | LBC on ResNet-50, 2:4 · ImageNet | Top-1 accuracy (%): 77.2 -> 72.9 | Hurt |
| Extend the gradual-pruning schedule from final epoch 0 to 30 (ablation) | LBC on ResNet-50, 1:4 · ImageNet | Top-1 accuracy (%): 75.2 -> 75.4 | Helped |
| Extend the gradual-pruning schedule to final epoch 60 (ablation) | LBC on ResNet-50, 1:4 · ImageNet | Top-1 accuracy (%): 75.2 -> 75.9 | Helped |
| Start gradual pruning at epoch 30 and end at epoch 60 (ablation) | LBC on ResNet-50, 1:4 · ImageNet | Top-1 accuracy (%): 75.2 -> 76.0 | Helped |
| Increase the gradual-pruning schedule from 0–0 to 0–60 (ablation) | LBC on ResNet-50, 1:4 · ImageNet | Training FLOPs: 0.48× -> 0.51× | Hurt |
| Use LBC instead of SR-STE for 2:4 sparse Faster R-CNN | Faster R-CNN · COCO object detection | mAP: 38.2 -> 38.5 | Helped |
| Use LBC instead of SR-STE for 2:4 sparse Mask R-CNN | Mask R-CNN · COCO instance segmentation | Mask mAP: 35.3 -> 35.4 | Helped |
| Use LBC instead of SR-STE for 2:4 sparse DeiT-small | DeiT-small · ImageNet | Top-1 accuracy (%): 79.6 -> 80.1 | Helped |
| Use LBC instead of SR-STE for 1:16 sparse ResNet-50 | ResNet-50 · ImageNet | Top-1 accuracy (%): 71.5 -> 71.8 | Helped |
Closest papers by meaning
- MaxQ: Multi-Axis Query for N:M Sparsity Network (CVPR 2024 · similarity 0.87)
- Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes (AAAI 2024 · similarity 0.86)
- Sparse maximal update parameterization: A holistic approach to sparse training dynamics (NeurIPS 2024 · similarity 0.86)
- Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMs (ICLR 2024 · similarity 0.85)
- TSENOR: Highly-Efficient Algorithm for Finding Transposable N:M Sparse Masks (NeurIPS 2025 · similarity 0.85)
- Scaling Laws for Sparsely-Connected Foundation Models (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.