Research map / Label-efficient and robust learning / Domain adaptation and generalization
Learning Frequency-Adapted Vision Foundation Model for Domain Generalized Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FADA, which applies Haar wavelet decomposition to frozen vision foundation model features and adapts low-frequency (content) and high-frequency (style, with instance normalization) tokens via low-rank adaptation for domain-generalized semantic segmentation.
Problem
Domain-generalized semantic segmentation (DGSS) requires keeping style-invariant, content-stable representations when adapting a frozen vision foundation model to unseen driving domains.
Approach
Decompose frozen VFM features with a Haar wavelet transform into low-frequency (content) and high-frequency (style) components, adapt each with separate low-rank token branches, and apply instance normalization to the high-frequency similarity maps to suppress domain-specific style.
Key results
- On the GTA5 source setting, FADA improves over REIN by 1.83, 1.54, 1.99 and 1.50 mIoU on C, B, M and S targets.
- Trained on Cityscapes with DINOv2, FADA reaches 65.12 mIoU on BDD, 75.86 on Mapillary, 63.78 on GTA5 and 49.75 on SYNTHIA, outperforming REIN by up to 1.83 mIoU.
- On ACDC adverse conditions (C source), FADA attains 71.5 mean mIoU vs 69.6 for REIN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply FADA instead of REIN | DINOv2 for DGSS · GTA5 → CityScapes | mIoU (%): 66.4 -> 68.23 | Helped |
| Apply FADA instead of REIN | DINOv2 for DGSS · GTA5 → BDD-100K | mIoU (%): 60.4 -> 61.94 | Helped |
| Apply FADA instead of REIN | DINOv2 for DGSS · GTA5 → Mapillary | mIoU (%): 66.1 -> 68.09 | Helped |
| Apply FADA instead of REIN | DINOv2 for DGSS · GTA5 → SYNTHIA | mIoU (%): 48.86† -> 50.36 | Helped |
| Apply FADA instead of REIN | DINOv2 for DGSS · SYNTHIA → unseen targets | mIoU (%): +1.45%, +1.41%, +1.22%, +1.29% | Helped |
| Apply FADA instead of REIN | DINOv2 for DGSS · CityScapes → unseen targets | mIoU (%): +1.58%, +1.83%, +1.37%, +1.19% | Helped |
| Apply FADA instead of REIN | DGSS across adverse conditions · CityScapes → ACDC Fog, Night, Rain, Snow | mIoU (%): +0.7%, +1.5%, +2.5%, +2.9% | Helped |
| Add low-frequency LL adaptation (ablation) | FADA · CityScapes → BDD-100K | mIoU (%): 62.43 -> 63.85 | Helped |
| Add low-frequency LL adaptation (ablation) | FADA · CityScapes → Mapillary | mIoU (%): 73.05 -> 74.16 | Helped |
| Add LH high-frequency adaptation (ablation) | FADA with LL · CityScapes → BDD-100K | mIoU (%): 63.85 -> 64.04 | Helped |
| Add HL high-frequency adaptation (ablation) | FADA with LL and LH · CityScapes → BDD-100K | mIoU (%): 64.04 -> 64.69 | Helped |
| Add HH high-frequency adaptation (ablation) | FADA with LL, LH and HL · CityScapes → BDD-100K | mIoU (%): 64.69 -> 65.12 | Helped |
Surprising findings
- Fine-tuning only the low-frequency component already yields about 1 mIoU gain on most settings, and frequency-adapter placement in shallow layers performed slightly better than deep layers.
Limitations
- Increases trainable parameters relative to REIN (about 6M extra), as acknowledged by the authors.
- Gains over REIN are modest (roughly 1-3 mIoU) and no error bars are reported.
- Evaluated only on driving-scene DGSS benchmarks with a Mask2Former decoder.
Experimental setup
Datasets: Cityscapes BDD-100K Mapillary SYNTHIA GTA5 ACDC
Benchmarks: Cityscapes BDD-100K Mapillary SYNTHIA GTA5 ACDC
Metrics: mIoU
Baselines: IBN IW Iternorm DRPC ISW GTR DIRL SHADE SAW WildNet AdvStyle SPC
Builds on: REIN LoRA Mask2Former DINOv2
Closest papers by meaning
- Stronger Fewer & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentation (CVPR 2024 · similarity 0.89)
- Exploring Semantic Consistency and Style Diversity for Domain Generalized Semantic Segmentation (AAAI 2025 · similarity 0.88)
- Collaborating Foundation Models for Domain Generalized Semantic Segmentation (CVPR 2024 · similarity 0.87)
- Style Blind Domain Generalized Semantic Segmentation via Covariance Alignment and Semantic Consistence Contrastive Learning (CVPR 2024 · similarity 0.87)
- Learning Content-Enhanced Mask Transformer for Domain Generalized Urban-Scene Segmentation (AAAI 2024 · similarity 0.87)
- Learning Generalized Segmentation for Foggy-Scenes by Bi-directional Wavelet Guidance (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.