Research map / Label-efficient and robust learning / Domain adaptation and generalization
Learning Generalized Segmentation for Foggy-Scenes by Bi-directional Wavelet Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes bi-directional wavelet guidance (BWG), a Haar-wavelet-based self-attention module inside a Mask2Former-style Swin-B segmenter, to generalize urban-scene segmentation to unseen foggy scenes using only clear source-domain training data.
Problem
Semantic segmentation that generalizes to arbitrary unseen foggy driving scenes, rather than only to foggy target domains seen during curriculum domain adaptation training.
Approach
Decomposes features with Haar wavelets so low-frequency components feed a content-enhancement self-attention while high-frequency components feed urban-style and fog-style de-correlation self-attentions (with instance normalization), integrated into mask-level Transformer decoders in a learnable way.
Key results
- Achieves 76.7 mIoU on ACDC-fog, 51.3 on Foggy Zurich, 54.2 on Foggy Driving and 77.4 on Foggy-CityScapes when trained only on Cityscapes, outperforming the second-best method by 6.8, 11.8, 7.2 and 9.6 points.
- Against curriculum domain adaptation methods trained with extra foggy data, it outperforms by at least 12.2 mIoU on ACDC-fog and 1.0 mIoU over CuDA-Net on Foggy Zurich.
- Ablation: content, style and fog encoders contribute gains; low-frequency shifts (S2C, F2C) contribute more than high-frequency shifts (C2S, C2F).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the BWG mechanism to Mask2Former | Mask2Former with Swin-B · ACDC-fog | mIoU: 73.3 -> 76.7 | Helped |
| Add the BWG mechanism to Mask2Former | Mask2Former with Swin-B · Foggy Zurich | mIoU: 49.4 -> 51.3 | Helped |
| Add the BWG mechanism to Mask2Former | Mask2Former with Swin-B · Foggy-driving | mIoU: 51.1 -> 54.2 | Helped |
| Add the BWG mechanism to Mask2Former | Mask2Former with Swin-B · Foggy-CityScapes | mIoU: 73.8 -> 77.4 | Helped |
| Add style encoder S to content encoder C (ablation) | BWG · ACDC-fog | mIoU: 73.4 -> 75.2 | Helped |
| Add style encoder S to content encoder C (ablation) | BWG · Foggy-driving | mIoU: 51.1 -> 52.9 | Helped |
| Add fog encoder F to content and style encoders (ablation) | BWG · ACDC-fog | mIoU: 75.2 -> 76.7 | Helped |
| Add fog encoder F to content and style encoders (ablation) | BWG · Foggy-driving | mIoU: 52.9 -> 54.2 | Helped |
| Add S2C, shifting LL from style branch S to content branch C (ablation) | BWG · ACDC-fog | mIoU: +1.0% mIoU | Helped |
| Add F2C, shifting LL from fog branch F to content branch C (ablation) | BWG · ACDC-fog | mIoU: +0.9% mIoU | Helped |
| Add C2S, shifting high-frequency components from content branch C to style branch S (ablation) | BWG · Foggy-driving | mIoU: +0.3% mIoU | Helped |
| Add C2F, shifting high-frequency components from content branch C to fog branch F (ablation) | BWG · Foggy-driving | mIoU: +0.3% mIoU | Helped |
Surprising findings
- Generalization-only training on clear Cityscapes outperforms several curriculum domain adaptation methods that use extra foggy training data on Foggy Driving.
Limitations
- The fog de-correlation encoder is data-driven rather than physics-driven.
- Some compared domain adaptation baselines are unavailable or lack reported results, so comparisons are partial (N/A entries).
Experimental setup
Datasets: Cityscapes Clear Cityscapes
Benchmarks: ACDC-fog Foggy Zurich Foggy Driving Foggy-CityScapes
Metrics: mIoU
Baselines: Mask2Former SegFormer RefineNet SAM-fine-tune SAM-SSA IBNet Iternorm SW ISW SHADE SAW WildNet
Builds on: Mask2Former Swin Transformer Haar wavelet pooling SAW ISW
Closest papers by meaning
- Learning Content-Enhanced Mask Transformer for Domain Generalized Urban-Scene Segmentation (AAAI 2024 · similarity 0.88)
- Learning Frequency-Adapted Vision Foundation Model for Domain Generalized Semantic Segmentation (NeurIPS 2024 · similarity 0.86)
- Parsing All Adverse Scenes: Severity-Aware Semantic Segmentation with Mask-Enhanced Cross-Domain Consistency (AAAI 2024 · similarity 0.85)
- Stronger Fewer & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentation (CVPR 2024 · similarity 0.85)
- Weakly Supervised Semantic Segmentation for Driving Scenes (AAAI 2024 · similarity 0.85)
- Semantic Segmentation on Raindrop Degraded Images Using Two-Stage Dual Teacher-Student Learning (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.