Research map / Architectures and efficiency / Pruning and efficient tuning
Combined Group and Exclusive Sparsity for Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add exclusive sparsity (1,2)-norm to group sparsity (combined CGES) | AlexNet on ImageNet-1K · ImageNet-1K | top-1 accuracy: CGES 58.56% vs L1 57.55% at similar params; similar or slightly worse than full network | Mixed |
| combined group + exclusive sparsity regularizer | LeNet on CIFAR-10 · CIFAR-10 | accuracy: +2.17% over L2 baseline with 13.72% fewer parameters | Helped |
| combined group + exclusive sparsity regularizer | simple CNN on MNIST · MNIST | accuracy: no accuracy reduction with 36.48% fewer parameters | No clear effect |
| combined group + exclusive sparsity regularizer | WRN-16-10 on CIFAR-100 · CIFAR-100 | accuracy: -1.15% vs full network at 51.22% params | Hurt |
| replace weight decay with CGES in iterative pruning (Han et al.) | iterative pruning on LeNet/CIFAR-10 and simple CNN/MNIST · CIFAR-10 and MNIST (10% params) | accuracy: CIFAR-10 78.97% vs 76.37% for pruning baseline; MNIST 99.16% vs 98.71% | Helped |
| add exclusive sparsity only (ES) at all layers | AlexNet on ImageNet-1K · ImageNet-1K | top-1 accuracy: ES 57.99% vs L2 59.89% at 60.41% params; ES 58.89% at 67.37% params | Mixed |
| exclusive sparsity regularization (ablation) | CIFAR-10 networks · CIFAR-10 accuracy vs parameter sparsity | accuracy: improves over L2 at low sparsity but degrades as sparsity increases | Mixed |
| group sparsity (filter-wise l2,1) | CIFAR-10 CNN · CIFAR-10 accuracy vs parameter sparsity | accuracy: performs worse than l1 in general but better in certain sparsity ranges | Mixed |
| increasing exclusivity mu_l with layer depth (ES-Increasing) vs constant mu=0.5 (ES-Constant) (ablation) | CIFAR-10 CNN · CIFAR-10 accuracy vs parameter sparsity | accuracy: ES-Increasing better than ES-Constant across all sparsity ranges | Helped |
| apply exclusive sparsity to conv layers vs FC layers vs both (ablation) | CIFAR-10 CNN · CIFAR-10 accuracy vs parameter sparsity | accuracy: improvements on all variants; conv-only best at low sparsity, FC-only better at high sparsity | Helped |
| exclusive sparsity regularizer (ablation) | CIFAR-10 CNN training · CIFAR-10 convergence speed | iterations to reach given error: reaches same error in fewer iterations than L2 | Helped |
Closest papers by meaning
- Uncorrelated Group LASSO (AAAI 2016 · similarity 0.89)
- Learning sparse neural networks via sensitivity-driven regularization (NeurIPS 2018 · similarity 0.89)
- Sparsifying Networks via Subdifferential Inclusion (ICML 2021 · similarity 0.88)
- Sparsity Winning Twice: Better Robust Generalization from More Efficient Training (ICLR 2022 · similarity 0.88)
- Transferable Structural Sparse Adversarial Attack Via Exact Group Sparsity Training (CVPR 2024 · similarity 0.87)
- Global Minimizers of $\ell^p$-Regularized Objectives Yield the Sparsest ReLU Neural Networks (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.