Research map / Architectures and efficiency / CNN design
Receptive Fields As Experts in Convolutional Neural Architectures
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Mixture of Receptive Fields (MoRF), treating convolutions of different kernel sizes as experts routed per input (hard or soft routing) and merged by re-parameterization for inference, integrated into ResNet and ConvNeXt.
Problem
Receptive field size in CNNs is manually chosen (e.g., 3x3 vs 7x7), and a single fixed size may not suit all inputs or layers; the goal is to let the network select receptive fields adaptively per input.
Approach
Treat convolutions with different kernel sizes as experts in a mixture-of-experts layer, with an attention-based router selecting them per input (hard top-k or soft weighted sum), then fuse the experts into one kernel via re-parameterization so inference cost stays close to a single-kernel convolution.
Key results
- ConvNeXt-T with MoRF (soft) reaches 82.6% top-1 on ImageNet-1K vs 82.1% for the baseline, with throughput 718.5 vs 774.7 imgs/s.
- ResNet-18 with MoRF (soft) improves top-1 from 70.4% to 74.6% at 20M params (vs 12M).
- UperNet with ConvNeXt-T + MoRF (soft) reaches 47.7 ms mIoU on ADE20K vs 46.7 for ConvNeXt-T.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MoRF soft routing to ConvNeXt-T | ConvNeXt-T · ImageNet-1K, 224×224 | Top-1 accuracy: 82.1 -> 82.6 | Helped |
| Add MoRF soft routing to ConvNeXt-S | ConvNeXt-S · ImageNet-1K, 224×224 | Top-1 accuracy: 83.1 -> 83.7 | Helped |
| Add MoRF soft routing to ConvNeXt-B | ConvNeXt-B · ImageNet-1K, 224×224 | Top-1 accuracy: 83.8 -> 84.2 | Helped |
| Add MoRF soft routing to ConvNeXt-B | ConvNeXt-B · ImageNet-1K, 384×384 | Top-1 accuracy: 85.1 -> 85.3 | Helped |
| Add MoRF soft routing to ConvNeXt-T | ConvNeXt-T with Mask R-CNN · COCO val2017 | APb: 46.2 -> 46.8 | Helped |
| Add MoRF soft routing to ConvNeXt-T | ConvNeXt-T with Mask R-CNN · COCO val2017 | APm: 41.7 -> 42.2 | Helped |
| Add MoRF soft routing to ConvNeXt-T | ConvNeXt-T with UperNet · ADE20K val | ms mIoU: 46.7 -> 47.7 | Helped |
| Use receptive field experts {3, 5, 7, 9, 11} instead of {3, 5, 7} (ablation) | ConvNeXt + MoRF · ImageNet-1K | Top-1 accuracy: 80.26/80.41 -> 80.43/80.62 | Helped |
| Use receptive field experts {3, 5, 7, 9, 11, 13, 15} instead of {3, 5, 7, 9, 11} (ablation) | ConvNeXt + MoRF · ImageNet-1K | Top-1 accuracy: 80.43/80.62 -> 80.39/80.59 | Mixed |
| Use larger-spaced receptive field experts {3, 7, 11, 15, 19} instead of {3, 5, 7, 9, 11} (ablation) | ConvNeXt + MoRF · ImageNet-1K | Top-1 accuracy: 80.43/80.62 -> 80.38/80.58 | Mixed |
| Use receptive field experts {3, 9, 15, 21, 27} instead of {3, 5, 7, 9, 11} (ablation) | ConvNeXt + MoRF · ImageNet-1K | Top-1 accuracy: 80.43/80.62 -> 80.41/80.62 | Mixed |
| Use unnormalized rather than sigmoid attention for soft routing (ablation) | ConvNeXt + MoRF · ImageNet-1K | Top-1 accuracy: 80.37 -> 80.62 | Helped |
Surprising findings
- Frequency analysis shows per-sample receptive field selection is close to random in most blocks, contrary to the common intuition that hybrids use convolutions early and attention late.
Limitations
- Only seven receptive field experts were explored due to compute limits; larger expert counts (e.g., MoE-scale) were not tested.
- Training remains resource-intensive, with training throughput declining rapidly as kernel size grows.
- Only CNN architectures (ResNet, ConvNeXt) were studied; transformer application is left to future work.
Experimental setup
Datasets: ImageNet-1K COCO ADE20K
Benchmarks: ImageNet-1K COCO val2017 ADE20K validation
Metrics: top-1 accuracy box AP mask AP mIoU throughput (imgs/s) parameters
Baselines: ResNet-18 ResNet-50 ConvNeXt-T ConvNeXt-S ConvNeXt-B Swin Transformer DY-ResNet SKNet-50 AS-MLP Focal Transformer DeiT UperNet
Builds on: ConvNeXt ResNet Mixture of Experts noisy top-k gating RepVGG-style re-parameterization dynamic convolution
Closest papers by meaning
- QuARF: Quality-Adaptive Receptive Fields for Degraded Image Perception (AAAI 2025 · similarity 0.87)
- Unveiling the Unseen: Identifiable Clusters in Trained Depthwise Convolutional Kernels (ICLR 2024 · similarity 0.85)
- KernelWarehouse: Rethinking the Design of Dynamic Convolution (ICML 2024 · similarity 0.85)
- Improving the Effective Receptive Field of Message-Passing Neural Networks (ICML 2025 · similarity 0.85)
- How to Train Neural Field Representations: A Comprehensive Study and Benchmark (CVPR 2024 · similarity 0.85)
- PICNN: A Pathway towards Interpretable Convolutional Neural Networks (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.