Research map / Architectures and efficiency / Pruning and efficient tuning
Learning N:M Fine-grained Structured Sparse Neural Networks From Scratch
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the SR-STE regularization term to vanilla STE (ablation) | ResNet18 and ResNet50 with 2:4 sparsity · ImageNet-1K val | Top-1 accuracy: 69.9 -> 71.2; 76.4 -> 77.0 | Helped |
| Use 4:8 rather than 2:4 sparsity at the same sparsity level (ablation) | ResNet50 SR-STE · ImageNet-1K val | Top-1 accuracy: 77.0 -> 77.4 | Helped |
| Use 2:8 rather than 1:4 sparsity at the same sparsity level (ablation) | ResNet50 SR-STE · ImageNet-1K val | Top-1 accuracy: 75.3 -> 76.2 | Helped |
| Use 2:4 SR-STE instead of dense training | Faster R-CNN R50 · COCO val-2017, 1x and 2x schedules | mAP: 37.4 -> 38.2; 38.4 -> 39.2 | Helped |
| Use 2:8 instead of 2:4 sparsity (ablation) | Faster R-CNN R50 SR-STE · COCO val-2017, 1x and 2x schedules | mAP: 38.2 -> 37.2; 39.2 -> 38.9 | Mixed |
| Use 2:4 SR-STE instead of dense training | Mask R-CNN R50 · COCO val-2017, 1x schedule | Box mAP: 38.2 -> 39.0 | Helped |
| Use 2:4 SR-STE instead of dense training | Mask R-CNN R50 · COCO val-2017, 1x schedule | Mask mAP: 34.7 -> 35.3 | Helped |
| Use 2:4 SR-STE rather than dense weights | RAFT · FlyingChairs val | EPE: 0.86 -> 0.88 | Hurt |
| Use 2:4 SR-STE rather than dense weights | Transformer · WMT'14 EN-DE test | BLEU: 27.31 -> 27.23 | Hurt |
| Set the SR-STE coefficient negative instead of positive (ablation) | ResNet18 with 2:4 sparsity · ImageNet-1K | Top-1 accuracy: 68.5% | Hurt |
| Use a gradient-based refinement term instead of the parameter-based SR-STE term (ablation) | ResNet18 with 2:4 sparsity · ImageNet | Top-1 accuracy: 69.9% -> 69.4% | Hurt |
| Use a sign-constant refinement term instead of vanilla STE (ablation) | ResNet18 with 2:4 sparsity · ImageNet | Top-1 accuracy: about 0.9% | Helped |
Closest papers by meaning
- DominoSearch: Find layer-wise fine-grained N:M sparse schemes from dense neural networks (NeurIPS 2021 · similarity 0.91)
- Dynamic Sparse Training with Structured Sparsity (ICLR 2024 · similarity 0.89)
- S-STE: Continuous Pruning Function for Efficient 2:4 Sparse Pre-training (NeurIPS 2024 · similarity 0.89)
- A Single-Step, Sharpness-Aware Minimization is All You Need to Achieve Efficient and Accurate Sparse Training (NeurIPS 2024 · similarity 0.88)
- Harnessing Manycore Processors with Distributed Memory for Accelerated Training of Sparse and Recurrent Models (AAAI 2024 · similarity 0.88)
- Accelerating Linear Recurrent Neural Networks for the Edge with Unstructured Sparsity (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.