Research map / Label-efficient and robust learning / Domain adaptation and generalization
SemiDAViL: Semi-supervised Domain Adaptation with Vision-Language Guidance for Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SemiDAViL, a language-guided semi-supervised domain adaptation framework for semantic segmentation that combines CLIP initialization, captioning-based dense language guidance, consistency training, and a dynamic class-balanced CE loss (DyCE).
Problem
Semi-supervised domain adaptation (SSDA) for semantic segmentation suffers from confusion among visually similar classes under limited target labels, source-biased learning, and class imbalance that hurts tail classes.
Approach
Initialize student-teacher segmenters with CLIP vision encoder, fuse vision features with dense language embeddings from BLIP-2 captions via a symmetric attention module (DLG), and reweight the supervised CE loss by mined-batch class frequency (DyCE) to counter class imbalance.
Key results
- On GTA5->Cityscapes with 100 labeled target samples, SemiDAViL reaches 71.1% mIoU, 1.6% above the previous best IIDM.
- In the fully unsupervised setting (0 labels) on GTA5->Cityscapes, the method reports 67.7% mIoU, about 5% above DIGA.
- On Synapse, adding DyCE to UA-MT raises mean DSC from 20.3% to 48.1% in the 20% labeled SSL setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DyCE to CT + VLP + DLG (ablation) | SemiDAViL · GTA5 → Cityscapes, 100 labeled target images | mIoU: 70.3% to 71.1% | Helped |
| add DyCE to CT + VLP + DLG (ablation) | SemiDAViL · Synthia → Cityscapes, 100 labeled target images | mIoU: 74.9% to 76.9% | Helped |
| add DyCE to consistency training (ablation) | SemiDAViL · GTA5 → Cityscapes, 100 labeled target images | mIoU: 54.5 to 63.3 | Helped |
| add DyCE to consistency training (ablation) | SemiDAViL · Synthia → Cityscapes, 100 labeled target images | mIoU: 60.2 to 68.7 | Helped |
| add vision-language pre-training (ablation) | SemiDAViL with CT · GTA5 → Cityscapes, 100 labeled target images | mIoU: 54.5 to 65.6 | Helped |
| add vision-language pre-training (ablation) | SemiDAViL with CT · Synthia → Cityscapes, 100 labeled target images | mIoU: 60.2 to 71.9 | Helped |
| add dense language guidance (ablation) | SemiDAViL with CT + VLP · GTA5 → Cityscapes, 100 labeled target images | mIoU: 65.6 to 70.3 | Helped |
| add dense language guidance (ablation) | SemiDAViL with CT + VLP · Synthia → Cityscapes, 100 labeled target images | mIoU: 71.9 to 74.9 | Helped |
| add DyCE (ablation) | UA-MT · Synapse, 20% labeled | Average DSC: 20.3% to 48.1(+27.9) | Helped |
| add DyCE (ablation) | DHC · Synapse, 20% labeled | Average DSC: 48.6% to 57.9% | Helped |
| add DyCE (ablation) | A&D · Synapse, 20% labeled | Average DSC: 60.9% to 65.5(+4.6) | Helped |
| use SemiDAViL with DyCE instead of IIDM | GTA5 → Cityscapes SSDA · GTA5 → Cityscapes, 100 labeled target images | 19-class mIoU: 69.5 to 71.1 | Helped |
Limitations
- Relies on an off-the-shelf captioning model (BLIP-2) to generate text descriptions for each image.
- Hyperparameters such as the hard-instance ratio and weight-balancing factor require tuning.
Experimental setup
Datasets: GTA5 SYNTHIA Cityscapes Synapse
Benchmarks: GTA5->Cityscapes Synthia->Cityscapes Synapse
Metrics: mIoU DSC ASD
Baselines: IIDM DSTC S-Depth DECOTA ALFSA SS-ADA DACS++ DaFormer DIGA A&D DHC UniMatch
Builds on: SemiVL CLIP DACS UniMatch BLIP-2
Closest papers by meaning
- Semi-Supervised Domain Adaptation Based on Dual-Level Domain Mixing for Semantic Segmentation (CVPR 2021 · similarity 0.93)
- Universal Domain Adaptation for Semantic Segmentation (CVPR 2025 · similarity 0.91)
- Balanced Learning for Domain Adaptive Semantic Segmentation (ICML 2025 · similarity 0.90)
- Semantic Library Adaptation: LoRA Retrieval and Fusion for Open-Vocabulary Semantic Segmentation (CVPR 2025 · similarity 0.89)
- Open-Set Domain Adaptation for Semantic Segmentation (CVPR 2024 · similarity 0.89)
- Exploring Semantic Consistency and Style Diversity for Domain Generalized Semantic Segmentation (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.