Research map / Architectures and efficiency / Pruning and efficient tuning
Minimum Variance Unbiased N:M Sparsity for the Neural Gradients
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace greedy minimum-MSE 1:2 pruning with MVUE 1:2 gradient pruning | ResNet18 · CIFAR-10 | accuracy: 85.5 -> 89.8 | Helped |
| Use MVUE 1:2 gradient pruning instead of dense gradients | ResNet18 · ImageNet | top-1 accuracy: 70.6 % -> 70.58 % | No clear effect |
| Use approximate MVUE 2:4 gradients instead of MVUE 1:2 gradients (ablation) | ResNet50 · ImageNet | top-1 accuracy: 76.4 % -> 77.12 % | Helped |
| Replace minimum-MSE greedy pruning with MVUE 1:2 | ResNet50 · ImageNet | top-1 accuracy: 59.3 % -> 76.4 % | Helped |
| Replace biased stochastic pruning with MVUE 1:2 (ablation) | ResNet18 · CIFAR-10 | accuracy: 71.8 -> 89.8 | Helped |
| Use uniform sampling instead of MVUE probabilities (ablation) | ResNet18 · CIFAR-10 | accuracy: 89.8 -> 85.8 | Hurt |
| Use unbiased uniform sampling instead of minimum-variance unbiased sampling (ablation) | ResNet18 · CIFAR-10 | accuracy: 89.8 -> 87.2 | Hurt |
| Add MVUE 1:2 gradient sparsity to transposable 2:4 weights (ablation) | ResNet50 · ImageNet | top-1 accuracy: 77.1 % -> 75.6 % | Hurt |
| Use approximate MVUE 2:4 rather than MVUE 1:2 gradients with sparse weights (ablation) | ResNet50 · ImageNet | top-1 accuracy: 75.6 % -> 77.1 % | Helped |
| Add 8-bit quantization to all-phase 2:4 sparsity (ablation) | ResNet18 · ImageNet | top-1 accuracy: 70.6 % -> 70.3 % | Hurt |
| Replace exact MVUE 2:4 mask computation with approximate MVUE 2:4 (ablation) | ResNet50 · overhead | overhead: 95 % -> 3 % | Helped |
| Reuse random samples every 50 iterations instead of sampling anew each iteration (ablation) | Approx-MVUE 2:4 · ResNet18 ImageNet | top-1 validation accuracy | No clear effect |
Closest papers by meaning
- Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks (NeurIPS 2021 · similarity 0.91)
- DominoSearch: Find layer-wise fine-grained N:M sparse schemes from dense neural networks (NeurIPS 2021 · similarity 0.90)
- Neural gradients are near-lognormal: improved quantized and sparse training (ICLR 2021 · similarity 0.89)
- Probabilistic Neural Pruning via Sparsity Evolutionary Fokker-Planck-Kolmogorov Equation (ICLR 2025 · similarity 0.88)
- How Sparse Can We Prune A Deep Network: A Fundamental Limit Perspective (NeurIPS 2024 · similarity 0.87)
- Effective Interplay between Sparsity and Quantization: From Theory to Practice (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.