Research map / Architectures and efficiency / Pruning and efficient tuning
Spartan: Differentiable Sparsity via Regularized Transportation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Spartan instead of Top-KAST | ResNet-50 · ImageNet-1K val, 95% sparsity, 100 epochs | top-1 accuracy: 73.51 ± 0.16 -> 74.68 ± 0.24 | Helped |
| Use Spartan instead of Top-KAST | ResNet-50 · ImageNet-1K val, 95% sparsity, 200 epochs | top-1 accuracy: 75.20 ± 0.11 -> 75.92 ± 0.01 | Helped |
| Use Spartan instead of Top-KAST | ResNet-50 · ImageNet-1K val, 95% sparsity, 400 epochs | top-1 accuracy: 75.90 ± 0.04 -> 76.48 ± 0.20 | Helped |
| Use Spartan instead of Top-KAST | ResNet-50 · ImageNet-1K val, 97.5% sparsity, 400 epochs | top-1 accuracy: 72.58 ± 0.13 -> 74.15 ± 0.10 | Helped |
| Use Spartan instead of Top-KAST | ViT-B/16 with 16×16 block pruning · ImageNet-1K val, 90% sparsity, 300 epochs | top-1 accuracy: 74.11 ± 0.20 -> 79.12 ± 0.16 | Helped |
| Use Spartan instead of Top-KAST | ViT-B/16 with 32×32 block pruning · ImageNet-1K val, 90% sparsity, 300 epochs | top-1 accuracy: 70.67 ± 0.96 -> 78.43 ± 0.10 | Helped |
| Use Spartan instead of Top-KAST | ViT-B/16 with unstructured pruning · ImageNet-1K val, 90% sparsity, 300 epochs | top-1 accuracy: 80.86 ± 0.03 -> 81.18 ± 0.04 | Helped |
| Omit hard top-k projection in the forward pass (ablation) | Spartan on ResNet-50 · ImageNet-1K val, 95% sparsity, 100 epochs | top-1 accuracy | Hurt |
| Increase βmax when training without dual averaging (ablation) | Spartan on ResNet-50 · ImageNet-1K val, 95% sparsity, 100 epochs | top-1 accuracy | Helped |
| Use βmax = 10 rather than the high-exploration or low-exploration endpoints (ablation) | Spartan on ResNet-50 · ImageNet-1K val, 95% sparsity | top-1 accuracy: 68.2% (IMP) -> 74.6% (βmax = 10); 73.5% (Top-KAST) | Helped |
| Train for more epochs | Spartan on ResNet-50 · ImageNet-1K val, sparse models | top-1 accuracy | Helped |
| Use valuation v0.5 instead of v1 for FLOP-sensitive pruning (ablation) | Spartan on ResNet-50 · ImageNet-1K val, 5% FLOP budget | top-1 accuracy: 73.88 -> 74.13 | Helped |
Closest papers by meaning
- Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes (AAAI 2024 · similarity 0.89)
- Sparse maximal update parameterization: A holistic approach to sparse training dynamics (NeurIPS 2024 · similarity 0.88)
- A Single-Step, Sharpness-Aware Minimization is All You Need to Achieve Efficient and Accurate Sparse Training (NeurIPS 2024 · similarity 0.88)
- Sparse Cocktail: Every Sparse Pattern Every Sparse Ratio All At Once (ICML 2024 · similarity 0.87)
- MaxQ: Multi-Axis Query for N:M Sparsity Network (CVPR 2024 · similarity 0.87)
- Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.