Research map / Optimization / Implicit bias and generalization
On Implicit Filter Level Sparsity in Convolutional Neural Networks
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add L2 regularization to Adam | BasicNet · CIFAR10 | filter sparsity by gamma (%): 0 -> 70 | Helped |
| Use Adam with L2 instead of SGD with L2 at comparable test error | BasicNet · CIFAR10 | filter sparsity by gamma (%): 0 -> 70 | Helped |
| Use L2 regularization instead of weight decay with Adam | BasicNet · CIFAR10 | filter sparsity by gamma (%): 40 -> 70 | Helped |
| Increase task difficulty from CIFAR10 to CIFAR100 | BasicNet with Adam and L2 1e-4 · CIFAR10 and CIFAR100 | filter sparsity by gamma (%): 70 -> 47 | Hurt |
| Increase batch size from 40 to 160 (ablation) | BasicNet with Adam and L2 1e-4 · CIFAR10 | filter sparsity by gamma (%): 70 -> 61 | Hurt |
| Increase batch size from 40 to 160 (ablation) | BasicNet with Adam and L2 1e-4 · CIFAR100 | filter sparsity by gamma (%): 46 -> 29 | Hurt |
| Reduce network width (ablation) | BasicNet with Adam and L2 1e-4 · CIFAR10 | filter sparsity by gamma (%): 70 -> 51 -> 23 | Hurt |
| Replace Adam with AMSGrad (ablation) | BasicNet with L2 1e-4 · CIFAR100 | filter sparsity by gamma (%): 47 -> 13 | Hurt |
| Replace ReLU with Leaky ReLU (negative slope 0.01) (ablation) | BasicNet with Adam and L2 1e-4 · CIFAR100 | filter sparsity by gamma (%): 47 -> 41 | Hurt |
| Reduce BatchNorm gamma learning rate by a factor of 10 (ablation) | BasicNet with Adam and L2 1e-4 · CIFAR100 | filter sparsity by gamma (%): 46 -> 33 | Hurt |
| Reduce BatchNorm beta learning rate by a factor of 10 (ablation) | BasicNet with Adam and L2 1e-4 · CIFAR100 | filter sparsity by gamma (%): 46 -> 51 | Helped |
| Initialize BatchNorm beta to -0.1 instead of 0 (ablation) | BasicNet with Adam and L2 1e-4 · CIFAR100 | filter sparsity by gamma (%): 46 -> 46 | No clear effect |
Closest papers by meaning
- DPFPS: Dynamic and Progressive Filter Pruning for Compressing Convolutional Neural Networks from Scratch (AAAI 2021 · similarity 0.90)
- RePr: Improved Training of Convolutional Filters (CVPR 2019 · similarity 0.90)
- Compressing Convolutional Neural Networks via Factorized Convolutional Filters (CVPR 2019 · similarity 0.90)
- Fast and Controllable Post-training Sparsity: Learning Optimal Sparsity Allocation with Global Constraint in Minutes (AAAI 2024 · similarity 0.87)
- Mask in the Mirror: Implicit Sparsification (ICLR 2025 · similarity 0.87)
- Effective Interplay between Sparsity and Quantization: From Theory to Practice (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.