Research map / Architectures and efficiency / Attention and transformers
LSNet: See Large, Focus Small
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LS (Large-Small) convolution, pairing large-kernel static perception with small-kernel dynamic grouped aggregation, and builds the LSNet family of lightweight vision networks that beats prior efficient CNN/ViT models on ImageNet, COCO, and ADE20K.
Problem
Lightweight vision networks rely on self-attention or static convolution for token mixing, which limits effectiveness and efficiency under tight computational budgets.
Approach
Decouple token mixing into large-kernel perception (large depth-wise static conv generating context-adaptive aggregation weights) and small-kernel aggregation (grouped dynamic convolution using those weights over a small neighborhood), inspired by the human peripheral/foveal vision split ("See Large, Focus Small").
Key results
- LSNet-B reaches 80.3% top-1 on ImageNet-1K at 1.3G FLOPs, outperforming AFFNet by 0.5% with nearly 3x faster inference.
- LS convolution improves LSNet-T top-1 by 2.3% over a variant without LS conv with only 0.02G FLOPs increase.
- LSNet-B achieves 43.0 mIoU on ADE20K with Semantic FPN, above FastViT-SA24 (41.0) and SwiftFormer-L1 (41.4).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LSNet-B instead of AFFNet | lightweight ImageNet-1K classification · ImageNet-1K | Top-1 accuracy: 79.8 -> 80.3 | Helped |
| Replace identity token mixing with LS convolution (ablation) | LSNet-T · ImageNet-1K | Top-1 accuracy: 69.3 -> 71.6 | Helped |
| Use LS convolution instead of alternative token mixers (ablation) | LSNet-T · ImageNet-1K | Top-1 accuracy: 70.8 -> 71.6; 70.6 -> 71.6; 70.5 -> 71.6; 69.7 -> 71.6; 70.5 -> 71.6 | Helped |
| Add large-kernel depth-wise convolution to LKP (ablation) | LSNet-T · ImageNet-1K | Top-1 accuracy: 70.5 -> 71.6 | Helped |
| Increase LKP kernel size from KL = 3 to KL = 5 (ablation) | LSNet-T · ImageNet-1K | Top-1 accuracy: 70.9 -> 71.2 | Helped |
| Increase LKP kernel size from KL = 5 to KL = 9 (ablation) | LSNet-T · ImageNet-1K | Top-1 accuracy: 71.2 -> 71.5 | Helped |
| Add adaptive small-kernel aggregation instead of static depth-wise convolution (ablation) | LSNet-T · ImageNet-1K | Top-1 accuracy: 70.1 -> 71.6 | Helped |
| Increase SKA kernel size from KS = 1 to KS = 3 (ablation) | LSNet-T · ImageNet-1K | Top-1 accuracy: 69.6 -> 71.6 | Helped |
| Increase SKA kernel size from KS = 3 to KS = 5 (ablation) | LSNet-T · ImageNet-1K | Top-1 accuracy: 71.6 -> 71.6 | No clear effect |
| Set channels per group C/G to 1 instead of the default 8 (ablation) | LSNet-T · ImageNet-1K | Top-1 accuracy: 71.6 -> 71.7 | No clear effect |
| Set channels per group C/G to 16 instead of the default 8 (ablation) | LSNet-T · ImageNet-1K | Top-1 accuracy: 71.6 -> 71.3 | Hurt |
| Set channels per group C/G to 32 instead of the default 8 (ablation) | LSNet-T · ImageNet-1K | Top-1 accuracy: 71.6 -> 70.9 | Hurt |
Surprising findings
- Replacing 3x3 convolutions in ResNet50 with LS convolution raises top-1 by 1.9% while reducing FLOPs from 4.1G to 2.6G.
Limitations
- Experiments on ablations use 100-epoch training due to compute limits, so absolute accuracies are below full-recipe results.
- Evaluation is limited to ImageNet classification, COCO detection/segmentation, and ADE20K segmentation; no other domains are tested.
Experimental setup
Datasets: ImageNet-1K COCO-2017 ADE20K
Benchmarks: ImageNet-1K COCO-2017 ADE20K ImageNet-C ImageNet-A ImageNet-R ImageNet-Sketch
Metrics: top-1 accuracy FLOPs throughput AP mIoU mCE
Baselines: EfficientViT RepViT FastViT AFFNet StarNet EdgeNeXt UniRepLKNet SHViT PoolFormer PVTv2
Builds on: MobileNet EfficientViT RepViT FastViT ConvNeXt
Closest papers by meaning
- Global Filter Networks for Image Classification (NeurIPS 2021 · similarity 0.87)
- UniRepLKNet: A Universal Perception Large-Kernel ConvNet for Audio Video Point Cloud Time-Series and Image Recognition (CVPR 2024 · similarity 0.87)
- KernelWarehouse: Rethinking the Design of Dynamic Convolution (ICML 2024 · similarity 0.87)
- AsCAN: Asymmetric Convolution-Attention Networks for Efficient Recognition and Generation (NeurIPS 2024 · similarity 0.86)
- ParameterNet: Parameters Are All You Need for Large-scale Visual Pretraining of Mobile Networks (CVPR 2024 · similarity 0.86)
- SparX: A Sparse Cross-Layer Connection Mechanism for Hierarchical Vision Mamba and Transformer Networks (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.