Research map / Detection and segmentation / CNN features
More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SLaK's 51×51 sparse factorized kernels instead of ConvNeXt kernels | ConvNeXt-T · ImageNet-1K val, 224×224 | Top-1 accuracy: 82.1 -> 82.5 | Helped |
| Use SLaK-B instead of ConvNeXt-B | ConvNeXt-B · ImageNet-1K val, 384×384 | Top-1 accuracy: 85.1 -> 85.5 | Helped |
| Use 51×51 SLaK-T instead of ConvNeXt-T | ConvNeXt-T · ADE20K val, 300-epoch pretraining / 160K finetuning | mIoU: 46.0 -> 47.6 | Helped |
| Use 51×51 SLaK-T instead of ConvNeXt-T | ConvNeXt-T · PASCAL VOC 2007 | mAP (%): 80.6 -> 82.7 | Helped |
| Use 51×51 SLaK-T instead of ConvNeXt-T | ConvNeXt-T · MS COCO val, 300-epoch pretraining / 3× finetuning | APbox: 50.4 -> 51.3 | Helped |
| Naively enlarge kernels to 31-29-27-13 (ablation) | ConvNeXt-T · ImageNet-1K, 120 epochs | Top-1 accuracy: 81.0 -> 80.5 | Hurt |
| Add structural re-parameterization for 31-29-27-13 kernels (ablation) | ConvNeXt-T · ImageNet-1K, 120 epochs | Top-1 accuracy: 81.0 -> 81.5 | Helped |
| Increase RepLKNet kernel sizes from 31-class to 51-class (ablation) | ConvNeXt-T with structural re-parameterization · ImageNet-1K, 120 epochs | Top-1 accuracy: 81.5 -> 81.3 | Hurt |
| Replace large square kernels with parallel rectangular M×5 and 5×M kernels (ablation) | ConvNeXt-T · ImageNet-1K, 120 epochs, 51-class kernels | Top-1 accuracy: 81.3 -> 81.5 | Helped |
| Apply 40% dynamic sparsity to decomposed kernels without widening (ablation) | ConvNeXt-T · ImageNet-1K, 120 epochs, 51-class kernels | Top-1 accuracy: 81.5 -> 80.5 | Hurt |
| Expand width 1.3× alongside 40% sparsity (ablation) | Decomposed ConvNeXt-T · ImageNet-1K, 120 epochs, 51-class kernels | Top-1 accuracy: 81.3 -> 81.6 | Helped |
| Increase width/sparsity setting from (0.40, 1.3×) to (0.55, 1.5×) (ablation) | SLaK-T · ImageNet-1K, 120 epochs | Top-1 accuracy: 81.6 -> 81.7 | Helped |
Closest papers by meaning
- PeLK: Parameter-efficient Large Kernel ConvNets with Peripheral Convolution (CVPR 2024 · similarity 0.90)
- UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio Video Point Cloud Time-Series and Image Recognition (CVPR 2024 · similarity 0.90)
- CAM Back Again: Large Kernel CNNs from a Weakly Supervised Object Localization Perspective (CVPR 2024 · similarity 0.89)
- Revealing the Dark Secrets of Extremely Large Kernel ConvNets on Robustness (ICML 2024 · similarity 0.89)
- KernelWarehouse: Rethinking the Design of Dynamic Convolution (ICML 2024 · similarity 0.88)
- InceptionNeXt: When Inception Meets ConvNeXt (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.