Research map / Detection and segmentation / CNN features
When Semantic Segmentation Meets Frequency Aliasing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes boundary segmentation errors through aliasing, proposing an equivalent sampling rate-based Nyquist frequency, an aliasing score, a de-aliasing filter (DAF) and a frequency mixing module (FreqMix) that improve semantic and low-light instance segmentation.
Problem
Hard pixels in semantic segmentation, especially at object boundaries, are poorly characterized; existing work only splits images into easy and hard regions and does not explain why hard boundary pixels arise.
Approach
Categorize boundary errors into false responses, merging mistakes and displacements, link them to aliasing via an equivalent sampling rate (accounting for kernel size and channel expansion) to compute the Nyquist frequency and an aliasing score, then remove above-Nyquist frequencies before downsampling (DAF) and adaptively mix low/high frequency components in encoder blocks (FreqMix).
Key results
- On Cityscapes with UPerNet-R50, DAF + FreqMix raises mIoU from 78.1 to 79.7 and lowers FErr/MErr/DErr from 27.2/25.1/26.9 to 25.2/23.3/24.8.
- On ADE20K, Ours (+DAF + FreqMix) improves UPerNet mIoU from 38.9 to 40.4.
- On low-light instance segmentation (LIS), the method gains +1.2 AP mask with PointRend (34.0 vs 32.8) and +1.1 with Mask2Former (36.7 vs 35.6).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DAF and FreqMix | UPerNet-R50 · Cityscapes validation | mIoU: 78.1 -> 79.7 | Helped |
| Add DAF and FreqMix | UPerNet · PASCAL VOC | mIoU: +1.8 mIoU | Helped |
| Add DAF and FreqMix | UPerNet · ADE20K | mIoU: +1.5 mIoU | Helped |
| Apply the proposed method | PointRend · LIS test | AP mask: +1.2 AP | Helped |
| Apply the proposed method | Mask2Former · LIS test | AP mask: +1.1 AP | Helped |
| Use a 3×3 blur kernel before downsampling (ablation) | UPerNet-R50 · Cityscapes validation | mIoU: 78.1 -> 78.8 | Helped |
| Increase the blur kernel from 3×3 to 5×5 (ablation) | UPerNet-R50 · Cityscapes validation | mIoU: 78.8 -> 78.7 | Hurt |
| Increase the blur kernel from 3×3 to 7×7 (ablation) | UPerNet-R50 · Cityscapes validation | mIoU: 78.8 -> 78.2 | Hurt |
| Set low-pass cut-off to 1/4 × √2 rather than 1/4 × 1.0 (ablation) | UPerNet-R50 with DAF · Cityscapes validation | mIoU: 78.6 -> 79.3 | Helped |
| Add channel-wise frequency weighting to spatial-wise FreqMix (ablation) | DAF + FreqMix · Cityscapes validation | mIoU: 79.3 -> 79.7; +0.4 mIoU | Helped |
| Predict frequency weights without decomposition (ablation) | DAF + FreqMix · Cityscapes validation | mIoU: 79.6 -> 79.7 | No clear effect |
Surprising findings
- A simple 3x3 Gaussian blur before downsampling improves mIoU, but larger blur kernels (5x5, 7x7) degrade performance.
- Adding Gaussian noise increases aliasing in the first stage, yet the measured aliasing ratio in deeper stages decreases, which the authors attribute to feature degradation rather than reduced aliasing.
Limitations
- The equivalent sampling rate is an empirical heuristic formula rather than a theoretically guaranteed one.
- Analysis and experiments focus on boundary errors in semantic segmentation; instance and panoptic segmentation are left for future work.
- Inference time increases substantially due to FFT/iFFT operations (45ms to 82ms).
Experimental setup
Datasets: Cityscapes PASCAL VOC ADE20K LIS COCO
Benchmarks: Cityscapes PASCAL VOC ADE20K LIS
Metrics: mIoU BIoU BAcc FErr MErr DErr AP mask AP box aliasing score
Baselines: UPerNet Blur AdaBlur FLC PointRend Mask2Former EnlightenGAN Zero-DCE SID REDI InternImage
Builds on: UPerNet Blur pooling (anti-aliasing) FLC PointRend Mask2Former
Closest papers by meaning
- FFR: Frequency Feature Rectification for Weakly Supervised Semantic Segmentation (CVPR 2025 · similarity 0.86)
- Frequency-Adaptive Dilated Convolution for Semantic Segmentation (CVPR 2024 · similarity 0.86)
- Lightweight Frequency Masker for Cross-Domain Few-Shot Semantic Segmentation (NeurIPS 2024 · similarity 0.85)
- MaSS13K: A Matting-level Semantic Segmentation Benchmark (CVPR 2025 · similarity 0.85)
- Hierarchy-Agnostic Unsupervised Segmentation: Parsing Semantic Image Structure (NeurIPS 2024 · similarity 0.85)
- Label-Efficient Few-Shot Semantic Segmentation with Unsupervised Meta-Training (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.