Research map / Label-efficient and robust learning / Domain adaptation and generalization
Exploring Semantic Consistency and Style Diversity for Domain Generalized Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SCSD, a CLIP-based Mask2Former framework for domain generalized semantic segmentation that combines text-guided semantic queries, text-driven Fourier style transformation, and a synergy-weighted style loss, reaching 49.11 mIoU averaged over four unseen domains when trained on GTAV.
Problem
Domain generalized semantic segmentation (DGSS) must generalize from source-domain data alone to unseen target domains; feature normalization confuses semantic features and domain randomization introduces domain-irrelevant noise.
Approach
Uses CLIP text embeddings to boost object queries in the mask decoder for cross-domain semantic consistency, and derives domain-difference text embeddings (specific minus general prompts) that are mapped to style features and used to modulate the low-frequency Fourier amplitude of image features, with a bank-based contrastive loss and adaptively weighted style aggregation loss to avoid domain coll
Key results
- Trained on GTAV, SCSD averages 49.11 mIoU over Cityscapes, BDD-100K, Mapillary and SYNTHIA, +4.08 over the prior state of the art.
- Under the G+S multi-source setting, SCSD reports 51.42 average mIoU over Cityscapes, BDD-100K and Mapillary.
- On ACDC adverse conditions (trained on GTAV), SCSD averages 35.66 mIoU versus 32.74 for FAMix.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SQB to the baseline (ablation) | SCSD · GTAV → Cityscapes, BDD-100K, Mapillary, SYNTHIA (Avg.) | mIoU (%): 46.41 -> 47.43 (+1.02) | Helped |
| Add TDST to the model with SQB (ablation) | SCSD · GTAV → Cityscapes, BDD-100K, Mapillary, SYNTHIA (Avg.) | mIoU (%): 47.43 -> 48.32 (+0.89) | Helped |
| Add SSO to the model with SQB and TDST (ablation) | SCSD · GTAV → Cityscapes, BDD-100K, Mapillary, SYNTHIA (Avg.) | mIoU (%): 48.32 -> 49.11 (+0.79) | Helped |
| Use style contrastive loss only (ablation) | SSO · GTAV → Cityscapes, BDD-100K, Mapillary, SYNTHIA (Avg.) | mIoU (%): 48.32 -> 48.38 | Helped |
| Use style aggregation loss only (ablation) | SSO · GTAV → Cityscapes, BDD-100K, Mapillary, SYNTHIA (Avg.) | mIoU (%): 48.32 -> 47.63 | Hurt |
| Use both SSO losses without synergy weighting (ablation) | SSO · GTAV → Cityscapes, BDD-100K, Mapillary, SYNTHIA (Avg.) | mIoU (%): 48.32 -> 48.76 | Helped |
| Add synergy weighting to both SSO losses (ablation) | SSO · GTAV → Cityscapes, BDD-100K, Mapillary, SYNTHIA (Avg.) | mIoU (%): 48.76 -> 49.11 | Helped |
| Apply Tanh constraint to original-feature style transformation (ablation) | TDST · GTAV → Cityscapes, BDD-100K, Mapillary, SYNTHIA (Avg.) | mIoU (%): 47.03 -> 48.14 (+1.11) | Helped |
| Use low-frequency image features instead of original features for style transformation, without Tanh (ablation) | TDST · GTAV → Cityscapes, BDD-100K, Mapillary, SYNTHIA (Avg.) | mIoU (%): 47.03 -> 47.99 | Helped |
| Apply Tanh constraint to low-frequency style transformation (ablation) | TDST · GTAV → Cityscapes, BDD-100K, Mapillary, SYNTHIA (Avg.) | mIoU (%): 47.99 -> 49.11 | Helped |
| Use SCSD instead of the strongest reported baseline | SCSD · GTAV → Cityscapes, BDD-100K, Mapillary, SYNTHIA (Avg.) | mIoU (%): 45.03 -> 49.11 (+4.08) | Helped |
| Use SCSD instead of the strongest reported baseline | SCSD · Cityscapes → BDD-100K, Mapillary, GTAV, SYNTHIA (Avg.) | mIoU (%): 48.40 -> 51.38 | Helped |
Surprising findings
- Only-style-aggregation loss degrades performance (47.63 vs 48.32 average), showing that aggregation without contrastive separation causes domain collapse.
Limitations
- Backbone is frozen due to limited GPU memory, which may limit adaptation capacity.
- FLOPs (648.7G) are higher than HGFormer and CMFormer, so computation is not uniformly lower.
Experimental setup
Datasets: GTAV SYNTHIA Cityscapes BDD-100K Mapillary ACDC
Benchmarks: Cityscapes BDD-100K Mapillary SYNTHIA GTAV ACDC
Metrics: mIoU
Baselines: IBN ISW Iternorm SHADE SAN-SAW WildNet HRDA TLDR DPCL HGFormer BlindNet FAMix
Builds on: Mask2Former CLIP WildNet FAMix
Closest papers by meaning
- Style Blind Domain Generalized Semantic Segmentation via Covariance Alignment and Semantic Consistence Contrastive Learning (CVPR 2024 · similarity 0.92)
- Collaborating Foundation Models for Domain Generalized Semantic Segmentation (CVPR 2024 · similarity 0.90)
- A Simple Recipe for Language-guided Domain Generalized Segmentation (CVPR 2024 · similarity 0.89)
- SemiDAViL: Semi-supervised Domain Adaptation with Vision-Language Guidance for Semantic Segmentation (CVPR 2025 · similarity 0.89)
- Generalize or Detect? Towards Robust Semantic Segmentation Under Multiple Distribution Shifts (NeurIPS 2024 · similarity 0.88)
- Learning Frequency-Adapted Vision Foundation Model for Domain Generalized Semantic Segmentation (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.