Research map / Label-efficient and robust learning / Semi-supervised learning
SemiVL: Semi-Supervised Semantic Segmentation with Vision-Language Guidance
Lukas Hoyer, David Joseph Tan, Muhammad Ferjad Naeem, Luc Van Gool, Federico Tombari
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace ImageNet initialization with CLIP vision-language pre-training (ablation) | UniMatch with ViT-B/16 · Pascal VOC (92 labels) | mIoU: 77.9 -> 80.6 | Improved |
| add spatial fine-tuning (attention layers only) (ablation) | SemiVL · Pascal VOC (92 labels) | mIoU: 80.6 -> 81.7 | Improved |
| add language-guided decoder (ablation) | SemiVL · Pascal VOC (92 labels) | mIoU: 81.7 -> 82.7 | Improved |
| add dense CLIP guidance from a frozen CLIP (ablation) | SemiVL · Pascal VOC (92 labels) | mIoU: 82.7 -> 83.2 | Improved |
| add dense CLIP guidance and class definitions (ablation) | SemiVL · Pascal VOC (1464 labels) | mIoU: 87.3 -> 87.4 -> 87.3 | No clear effect |
| add class definition guidance from annotation guidelines (ablation) | SemiVL · Pascal VOC (92 labels) | mIoU: 83.2 -> 84.0 | Improved |
| spatial fine-tuning instead of full fine-tuning, LoRA, prompt tuning or frozen (ablation) | SemiVL encoder · Pascal VOC (92 labels) | mIoU: 80.6 -> 81.7 (vs full fine-tuning) | Improved |
| decouple spatial and semantic reasoning instead of joint reasoning over all classes (ablation) | language-guided decoder · Pascal VOC (92 labels) | mIoU: 64.9 -> 82.7 | Improved |
| add semantic reasoning module on top of upsampling and spatial reasoning (ablation) | language-guided decoder · Pascal VOC (92 labels) | mIoU: 80.3 -> 82.7 | Improved |
| use guideline concepts with max aggregation instead of dictionary text, GPT concepts, or averaged embeddings (ablation) | class definition guidance · Pascal VOC (92 labels) | mIoU: 83.4 -> 84.0 (Guidelines Avg vs Max) | Improved |
| raise initial CLIP guidance weight from 0.1 to 1 (ablation) | dense CLIP guidance · Pascal VOC (92 labels) | mIoU: 84.0 -> 75.7 | Hurt |
| lower CLIP guidance confidence threshold from 0.9 to 0.7 (ablation) | dense CLIP guidance · Pascal VOC (92 labels) | mIoU: 84.0 -> 81.5 | Hurt |
Later papers that compared with this method
2 later accepted papers report SemiVL as a baseline in their results tables; 2 of them report a better number on most of the results they share with it.
- SSR-SAM: Retrieval-Style Segment Anything Model for Semi-Supervised Ultra-High-Resolution Image Segmentation (AAAI 2026 · better on 2 of 2 shared results)
- Weighting Pseudo-Labels via High-Activation Feature Index Similarity and Object Detection for Semi-Supervised Segmentation (ECCV 2024 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- SemiDAViL: Semi-supervised Domain Adaptation with Vision-Language Guidance for Semantic Segmentation (CVPR 2025 · similarity 0.92)
- Segmenting Visuals With Querying Words: Language Anchors For Semi-Supervised Image Segmentation (ECCV 2026 · similarity 0.92)
- Semi-Supervised Semantic Segmentation With High- and Low-Level Consistency (TPAMI 2019 · similarity 0.91)
- SemiCVT: Semi-Supervised Convolutional Vision Transformer for Semantic Segmentation (CVPR 2023 · similarity 0.91)
- Semi-Supervised Semantic Segmentation under Label Noise via Diverse Learning Groups (ICCV 2023 · similarity 0.90)
- Semi-Supervised Semantic Segmentation With Pixel-Level Contrastive Learning From a Class-Wise Memory Bank (ICCV 2021 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
