Research map / Architectures and efficiency / CNN design
On the Connection between Local Attention and Dynamic Depth-wise Convolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace local attention with static depth-wise convolution | Swin-style networks · ImageNet-1K val, tiny and base | top-1 accuracy: 81.3 -> 81.3; 83.3 -> 83.2 | No clear effect |
| Replace local attention with static depth-wise convolution | Swin-style networks · COCO 2017 val, tiny and base | APbox: 50.5 -> 49.9; 51.9 -> 51.1 | Hurt |
| Replace local attention with static depth-wise convolution | Swin-style networks · ADE20K val, tiny and base | mIoU: 44.5 -> 45.5; 48.1 -> 48.3 | Mixed |
| Add homogeneous dynamic weights to DWNet (ablation) | DWNet · ImageNet-1K, tiny | top-1 accuracy: 81.3 -> 81.9 | Helped |
| Add inhomogeneous dynamic weights to DWNet (ablation) | DWNet · ImageNet-1K, tiny | top-1 accuracy: 81.3 -> 81.8 | Helped |
| Use inhomogeneous rather than attention-based dynamic weights (ablation) | tiny local spatial-mixing network · ImageNet-1K | top-1 accuracy: 81.4 -> 81.8 | Helped |
| Use sliding rather than shifted-window sampling for attention (ablation) | Local MLP with attention · ImageNet-1K, tiny | top-1 accuracy: 81.3 -> 81.4 | No clear effect |
| Share static local weights across channels (ablation) | Local MLP · ImageNet-1K, tiny | top-1 accuracy: 80.2 -> 80.3 | No clear effect |
| Share depth-wise convolution weights across positions (ablation) | DWNet · ImageNet-1K, tiny | top-1 accuracy: 80.3 -> 81.3 | Helped |
| Add an extra depth-wise convolution before FFN (ablation) | i-dynamic DWNet · ImageNet-1K, tiny | top-1 accuracy: 81.8 -> 82.3; improved by 0.5 | Helped |
Closest papers by meaning
- Role of Locality and Weight Sharing in Image-Based Tasks: A Sample Complexity Separation between CNNs, LCNs, and FCNs (ICLR 2024 · similarity 0.88)
- Bridging the Divide: Reconsidering Softmax and Linear Attention (NeurIPS 2024 · similarity 0.87)
- FasterViT: Fast Vision Transformers with Hierarchical Attention (ICLR 2024 · similarity 0.87)
- Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials (NeurIPS 2025 · similarity 0.87)
- LF-ViT: Reducing Spatial Redundancy in Vision Transformer for Efficient Image Recognition (AAAI 2024 · similarity 0.87)
- KernelWarehouse: Rethinking the Design of Dynamic Convolution (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.