Research map / Architectures and efficiency / CNN design
Frequency Dynamic Convolution for Dense Image Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Frequency Dynamic Convolution (FDConv), which builds diverse parallel kernels by splitting a fixed Fourier-domain parameter budget into disjoint frequency groups, plus kernel spatial and frequency band modulation, for dense prediction with little parameter overhead.
Problem
Dynamic convolution uses multiple parallel weights, but their frequency responses are highly similar, so parameter cost grows with little adaptability gain; the paper targets dense image prediction (detection, segmentation) and classification.
Approach
Learn a fixed parameter budget in the Fourier domain, divided into frequency-disjoint groups to form n>10 frequency-diverse weights without extra parameters; add element-wise kernel spatial modulation (KSM) and spatially variant frequency band modulation (FBM) driven by local content.
Key results
- On COCO with Faster R-CNN, FDConv adds +3.6M parameters and reaches 39.4 APbox vs 37.2 baseline, beating ODConv (+65.1M, 39.2).
- On ADE20K with UPerNet (ResNet-50), FDConv reaches 43.8 mIoU SS at 70M params vs ODConv 43.3 at 131M and KW 43.5 at 141M.
- Mask2Former-ResNet-50 on Cityscapes improves from 79.4 to 80.4 mIoU with FDConv.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the standard convolution with FDConv | Faster R-CNN · COCO val | APbox: 38.3 -> 39.4 | Helped |
| Replace the standard convolution with FDConv | Mask R-CNN · COCO val | APbox: 39.6 -> 42.4 | Helped |
| Replace the standard convolution with FDConv | Mask R-CNN · COCO val | APmask: 36.4 -> 38.6 | Helped |
| Replace the standard convolution with FDConv | UPerNet, ResNet-50 · ADE20K val | mIoU (SS): 40.7 -> 43.8 | Helped |
| Replace the standard convolution with FDConv | UPerNet, ResNet-50 · ADE20K val | mIoU (MS): 41.8 -> 44.9 | Helped |
| Add FDConv | Mask R-CNN, ConvNeXt-T · COCO | APbox: 43.4 -> 45.2 | Helped |
| Add FDConv | Mask R-CNN, ConvNeXt-T · COCO | APmask: 39.7 -> 40.8 | Helped |
| Add FDConv | Mask R-CNN, Swin-T · COCO | APbox: 42.7 -> 44.5 | Helped |
| Add FDConv | Mask R-CNN, Swin-T · COCO | APmask: 39.3 -> 40.5 | Helped |
| Add FDConv | Mask2Former, ResNet-50 · Cityscapes | mIoU: 79.4 -> 80.4 (+1.0) | Helped |
| Add FDConv | Mask2Former, Swin-B† · ADE20K | mIoU: 53.9 -> 54.9 (+1.0) | Helped |
| Add FDConv | MaskDINO, Swin-L† · ADE20K | mIoU: 56.6 -> 57.2 (+0.5) | Helped |
Surprising findings
- FDConv's FLOPs overhead (+1.8G) is larger than several baselines despite its small parameter increase, which the paper does not discuss in depth.
Limitations
- Parameter-efficient claims rely on a fixed budget; FLOPs overhead (+1.8G) is substantial relative to some baselines.
- Gains on heavy models are modest (+0.5 to +1.0 mIoU).
- Default settings (64 weights, four frequency bands) were chosen empirically.
Experimental setup
Datasets: COCO ADE20K Cityscapes
Benchmarks: COCO ADE20K Cityscapes
Metrics: AP mIoU parameters FLOPs cosine similarity
Baselines: CondConv DY-Conv DCD ODConv KW PEDConv
Builds on: DY-Conv CondConv ODConv KW Mask2Former
Closest papers by meaning
- Frequency-Adaptive Dilated Convolution for Semantic Segmentation (CVPR 2024 · similarity 0.89)
- Omni-Dimensional Dynamic Convolution (ICLR 2022 · similarity 0.89)
- Fast Fourier Convolution (NeurIPS 2020 · similarity 0.88)
- KernelWarehouse: Rethinking the Design of Dynamic Convolution (ICML 2024 · similarity 0.87)
- Beyond Spatial Domain: Cross-domain Promoted Fourier Convolution Helps Single Image Dehazing (AAAI 2025 · similarity 0.85)
- FlashFFTConv: Efficient Convolutions for Long Sequences with Tensor Cores (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.