Research map / Architectures and efficiency / CNN design
FreeNet: Liberating Depth-Wise Separable Operations for Building Faster Mobile Vision Architectures
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FreeNet, a family of separable-operator-free efficient vision backbones built from an S2-Mixer token mixer (sparse-sampled continuous convolutions) and a ShiftFFN channel mixer, achieving better accuracy-speed tradeoffs on ImageNet-1K and dense tasks.
Problem
Depth-wise separable operators (e.g., DWConv) are cheap in FLOPs and parameters but run slowly on real devices due to discontinuous memory access, and FFN channel expansion is a speed bottleneck in efficient backbones.
Approach
Replace separable token mixing with sparse-sampling convolutions (SWConv) that operate on a continuous subset of channels and uniformly dilated spatial pixels, and replace the standard FFN with ShiftFFN, which uses a lower 2x expansion plus a shift neck that cheaply generates globally-biased features concatenated to reach 4x width.
Key results
- On ImageNet-1k, FreeNet-S2 with advanced training reaches 78.8% top-1, outperforming StarNet-S4 (78.4%) by 0.4% while giving about 40% higher GPU throughput.
- Under basic settings FreeNet-S2 (77.0%) and FreeNet-S4 (79.4%) surpass FasterNet-T1 (76.2%) and FasterNet-T2 (78.9%) by 0.8% and 0.5%.
- On COCO Mask-RCNN, FreeNet-S4 reaches 39.1 box AP vs 38.0 for ResNet-50 and 38.9 for FastViT-SA12, with latency about 10% lower than FastViT-SA12.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FreeNet-S4 instead of SwiftFormer-L1 | ImageNet-1K classification · ImageNet-1K | Top-1 accuracy: 80.9 -> 81.0 | Helped |
| Use FreeNet-S2 instead of StarNet-S4 | ImageNet-1K classification · ImageNet-1K | Top-1 accuracy: 78.4 -> 78.8 | Helped |
| Use FreeNet-S4 instead of FastViT-SA12 | dense prediction · MS-COCO detection and instance segmentation | AP b: 38.9 -> 39.1 | Helped |
| Replace final-stage FFNs with ShiftFFNs (ablation) | ConvNeXt-T · ImageNet-1K | Top-1 accuracy: 82.119 -> 82.123 | No clear effect |
| Replace final-stage 7×7 DWConvs with S2-Mixers (ablation) | ConvNeXt-T · ImageNet-1K | Top-1 accuracy: 82.119 -> 82.131 | Helped |
| Replace final-stage ConvFFNs with ShiftFFNs (ablation) | PVTv2-b2-li · ImageNet-1K | Top-1 accuracy: 82.092 -> 82.089 | No clear effect |
| Replace final-stage linear self-attention with S2-Mixers (ablation) | PVTv2-b2-li · ImageNet-1K | Top-1 accuracy: 82.092 -> 82.126 | Helped |
| Replace last-stage mixers with ShiftFFN and S2-Mixer and increase depth (ablation) | PVTv2-b2-li · ImageNet-1K | Top-1 accuracy: 82.092 -> 83.259 | Helped |
Surprising findings
- Replacing the 7x7 DWConv in ConvNeXt-T's last stage with S2-Mixers slightly improved top-1 accuracy (82.119 to 82.131) while reducing latency only modestly, indicating separable operators are not necessary in late stages.
Limitations
- Gains over baselines are small in accuracy (often under 1%) and depend on training recipe.
- Speed measurements are hardware-specific (RTX-2080Ti, Snapdragon 8cx Gen 3), so the advantage may not transfer to other devices.
- Many of the training details and extra results are deferred to an appendix not included in the provided text.
Experimental setup
Datasets: ImageNet-1k MS-COCO ADE20K
Benchmarks: ImageNet-1k MS-COCO ADE20K
Metrics: top-1 accuracy GPU throughput GPU latency mobile GPU latency mobile CPU latency box AP mask AP mIoU FLOPs parameters
Baselines: StarNet FasterNet MobileOne SwiftFormer FastViT MobileViT EdgeNeXt EfficientNet
Builds on: FasterNet ConvNeXt PVTv2 MetaFormer
Closest papers by meaning
- EfficientViM: Efficient Vision Mamba with Hidden State Mixer based State Space Duality (CVPR 2025 · similarity 0.85)
- Efficient Modulation for Vision Networks (ICLR 2024 · similarity 0.85)
- IFORMER: INTEGRATING CONVNET AND TRANSFORMER FOR MOBILE APPLICATION (ICLR 2025 · similarity 0.84)
- MobileMamba: Lightweight Multi-Receptive Visual Mamba Network (CVPR 2025 · similarity 0.84)
- DEPrune: Depth-wise Separable Convolution Pruning for Maximizing GPU Parallelism (NeurIPS 2024 · similarity 0.84)
- LSNet: See Large, Focus Small (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.