Research map / Architectures and efficiency / CNN design
ShiftwiseConv: Small Convolutional Kernel with Large Kernel Effect
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Shiftwise (SW) convolution, which emulates large-kernel (e.g., 51x5 strip) convolutions with standard 3x3 group convolutions, shift-based multi-path feature fusion and coarse-grained pruning, yielding a pure CNN that beats SLaK and UniRepLKNet on ImageNet, COCO, ADE20K and nuScenes.
Problem
Large-kernel CNNs show diminishing returns when kernels are simply enlarged; the paper asks whether long-distance dependencies can be modeled through alternative mechanisms and whether small convolutions can replace large kernels.
Approach
Decouple large-kernel effects into fine-grained feature extraction via 3x3 convolutions (group conv with channel-shift fusion across multiple paths) plus sparse, connection-centric long-range dependency modeling, replacing SLaK's M×N strip convolutions with 3×3 operations and coarse-grained filter pruning.
Key results
- SW-tiny reaches 83.4% top-1 on ImageNet-1K at 31M params and 5.0G FLOPs, versus 83.2% for UniRepLKNet-T and 82.5% for SLaK-T.
- SW-small reaches 83.9% top-1 on ImageNet-1K, matching UniRepLKNet-S (83.9%) and exceeding SLaK-S (83.8%).
- Replacing SLaK's 51x5 strip convolutions with 3x3 standard convolutions plus shift fusion keeps accuracy near SLaK at 120 epochs (81.60% vs 81.6%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace SLaK's large strip convolutions with laid-out standard convolutions (ablation) | SLaK tiny · ImageNet-1K | accuracy: 82.5 -> 82.5 | No clear effect |
| Add two reparameterization branches (ablation) | SW · ImageNet-1K, 120 epochs | accuracy: 81.26 -> 81.52 | Helped |
| Move batch normalization after the SW branches (ablation) | SW with two Rep branches · ImageNet-1K, 120 epochs | accuracy: 81.52 -> 81.65 | Helped |
| Reduce convolution kernel size from 5 to 3 (ablation) | SW · ImageNet-1K, 120 epochs | accuracy: 81.34 -> 81.44 | Helped |
| Add another edge for channel-order mapping (ablation) | SW · ImageNet-1K, 120 epochs | accuracy: 81.26 -> 81.51 | Helped |
| Combine 2 Rep branches with 4 SW edges (ablation) | SW · ImageNet-1K, 120 epochs | accuracy: 81.51 -> 81.82 | Helped |
| Average initial sparsity across Rep branches (ablation) | SW with 2 Rep branches and 4 edges · ImageNet-1K, 120 epochs | accuracy: 81.82 -> 81.94 | Helped |
| Reduce sparse-mask sharing frequency (ablation) | SW · ImageNet-1K, 120 epochs | accuracy: 81.93 -> 81.95 | Helped |
| Use UniRepLKNet architectural hyperparameters (ablation) | SW · ImageNet-1K, 120 epochs | accuracy: 81.6 -> 82.25 | Helped |
| Add squeeze-and-excitation (ablation) | SW · ImageNet-1K, 120 epochs | accuracy: 82.25 -> 82.27 | Helped |
| Remove large convolution kernels in the first stage (ablation) | SW · ImageNet-1K, 120 epochs | accuracy: 82.27 -> 82.17 | Hurt |
| Use randomly generated subsets of masks for Rep branches (ablation) | SW · ImageNet-1K, 120 epochs | accuracy: 82.27 -> 81.72 | Hurt |
Surprising findings
- Replacing large kernels with 3x3 convolutions (N=5 to N=3) slightly improved accuracy rather than hurting it (81.44 to 81.34 range in ablation, with the smaller kernel benefiting the model).
- Increasing depth under a fixed parameter budget (UniRepLKNet's architecture) gave better results than widening the network.
Limitations
- Many design choices were tuned on 120-epoch ImageNet runs and then extended to 300 epochs, so hyperparameters are tailored to the benchmark.
- Some analysis findings (e.g., pruning patterns) were not clearly explained by the authors.
Experimental setup
Datasets: ImageNet-1K COCO ADE20K nuScenes
Benchmarks: ImageNet-1K COCO ADE20K nuScenes
Metrics: top-1 accuracy box AP mask AP mIoU mAP Params FLOPs
Baselines: SLaK UniRepLKNet ConvNeXt RepLKNet Swin Transformer FastViT CoAtNet PVTv2 HorNet InternImage FCOS3D
Builds on: SLaK UniRepLKNet RepLKNet ConvNeXt RepVGG
Closest papers by meaning
- Approximation and Learning with Deep Convolutional Models: a Kernel Perspective (ICLR 2022 · similarity 0.85)
- FlexConv: Continuous Kernel Convolutions With Differentiable Kernel Sizes (ICLR 2022 · similarity 0.84)
- Efficient Spatially-Variant Convolution via Differentiable Sparse Kernel Complex (ICLR 2026 · similarity 0.84)
- KernelWarehouse: Rethinking the Design of Dynamic Convolution (ICML 2024 · similarity 0.83)
- UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio Video Point Cloud Time-Series and Image Recognition (CVPR 2024 · similarity 0.83)
- Stochastic Kernel Regularisation Improves Generalisation in Deep Kernel Machines (NeurIPS 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.