Research map / Label-efficient and robust learning / Domain adaptation and generalization
Style Blind Domain Generalized Semantic Segmentation via Covariance Alignment and Semantic Consistence Contrastive Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BlindNet, which trains a segmentation encoder with covariance alignment (CML and CCL) and a decoder with semantic consistency contrastive learning (CWCL and SDCL) for single-source domain generalized semantic segmentation without extra modules or datasets.
Problem
Semantic segmentation models trained on a source domain (e.g., GTAV) degrade on unseen target domains due to style (texture) shifts; existing feature-normalization DGSS methods remove style but also remove content information.
Approach
Instead of removing style statistics, align encoder covariance matrices between an original image and its color-jittered augmentation while keeping cross-covariance diagonals at one to preserve content; in the decoder, apply InfoNCE-based class-wise and misclassification-targeted contrastive learning on augmented-image features.
Key results
- Trained on GTAV with ResNet-50, BlindNet reaches 45.72 (Cityscapes), 41.32 (BDD-100K), 47.08 (Mapillary) mIoU, the best among compared DGSS methods.
- Ablation on GTAV with ResNet-50 shows the full loss combination improves Cityscapes mIoU from 28.95 baseline to 45.72.
- BlindNet adds no parameters or inference cost over the DeepLabV3+ baseline (45.08M params, 277.78 GFLOPS vs 277.77).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BlindNet instead of DPCL | DeepLabV3+ with MobileNetV2 trained on GTAV · Cityscapes val | mIoU (%): 37.57 -> 37.66 | No clear effect |
| Use BlindNet instead of DPCL | DeepLabV3+ with MobileNetV2 trained on GTAV · BDD-100K val | mIoU (%): 35.45 -> 36.10 | Helped |
| Use BlindNet instead of DPCL | DeepLabV3+ with MobileNetV2 trained on GTAV · Mapillary val | mIoU (%): 40.30 -> 40.40 | No clear effect |
| Add covariance matching loss (CML) (ablation) | DeepLabV3+ with ResNet-50 trained on GTAV · Cityscapes, BDD-100K, and Mapillary | mIoU (%): 28.95 -> 38.08; 25.14 -> 36.65; 28.18 -> 40.62 | Helped |
| Add cross-covariance loss (CCL) to CML (ablation) | DeepLabV3+ with ResNet-50 trained on GTAV · Cityscapes, BDD-100K, and Mapillary | mIoU (%): 38.08 -> 40.42; 36.65 -> 37.81; 40.62 -> 43.91 | Helped |
| Add CWCL to CML (ablation) | DeepLabV3+ with ResNet-50 trained on GTAV · Cityscapes, BDD-100K, and Mapillary | mIoU (%): 38.08 -> 42.03; 36.65 -> 38.27; 40.62 -> 44.02 | Helped |
| Add SDCL to CML + CWCL (ablation) | DeepLabV3+ with ResNet-50 trained on GTAV · Cityscapes, BDD-100K, and Mapillary | mIoU (%): 42.03 -> 43.17; 38.27 -> 38.23; 44.02 -> 44.84 | Mixed |
| Replace diagonal-only CCL with whitening (ablation) | BlindNet · Cityscapes, BDD-100K, and Mapillary | mIoU (%): 40.42 -> 38.68; 37.81 -> 36.91; 43.91 -> 42.12 | Hurt |
| Increase CWCL classes sampled per image from 10 to 15 (ablation) | BlindNet · Cityscapes, BDD-100K, and Mapillary | mIoU (%): 45.57 -> 45.72; 38.88 -> 41.32; 46.37 -> 47.08 | Mixed |
| Increase CWCL negative samples per class from 10 to 50 (ablation) | BlindNet · Cityscapes, BDD-100K, and Mapillary | mIoU (%): 44.76 -> 45.72; 38.21 -> 41.32; 46.46 -> 47.08 | Helped |
| Increase CWCL negative samples per class from 50 to 100 (ablation) | BlindNet · Cityscapes, BDD-100K, and Mapillary | mIoU (%): 45.72 -> 44.44; 41.32 -> 39.14; 47.08 -> 46.29 | Hurt |
| Share CWCL's projection head with SDCL instead of using an individual head (ablation) | BlindNet · Cityscapes, BDD-100K, and Mapillary | mIoU (%): 44.91 -> 45.72; 38.45 -> 41.32; 46.31 -> 47.08 | Helped |
Surprising findings
- Using whitening (full covariance matching) instead of diagonal-only cross-covariance reduces performance (Cityscapes 38.68 vs 40.42 without the ablation's other terms), indicating removing non-diagonal content components hurts.
Limitations
- Authors note that the covariance alignment method needs improvement and stabilization.
- Evaluated only with DeepLabV3+ segmentation architecture on urban driving datasets.
Experimental setup
Datasets: GTAV Cityscapes SYNTHIA BDD-100K Mapillary ImageNet
Benchmarks: Cityscapes BDD-100K Mapillary SYNTHIA
Metrics: mIoU parameters GFLOPS inference time
Baselines: Baseline (DeepLabV3+) IBN-Net RobustNet SiamDoGe DIRL WildNet SANSAW SPC DPCL
Builds on: DeepLabV3+ RobustNet InfoNCE WildNet DPCL
Closest papers by meaning
- Exploring Semantic Consistency and Style Diversity for Domain Generalized Semantic Segmentation (AAAI 2025 · similarity 0.92)
- Collaborating Foundation Models for Domain Generalized Semantic Segmentation (CVPR 2024 · similarity 0.87)
- Learning Frequency-Adapted Vision Foundation Model for Domain Generalized Semantic Segmentation (NeurIPS 2024 · similarity 0.87)
- A Simple Recipe for Language-guided Domain Generalized Segmentation (CVPR 2024 · similarity 0.87)
- Generalize or Detect? Towards Robust Semantic Segmentation Under Multiple Distribution Shifts (NeurIPS 2024 · similarity 0.86)
- FisherTune: Fisher-Guided Robust Tuning of Vision Foundation Models for Domain Generalized Segmentation (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.