Research map / Label-efficient and robust learning / Domain adaptation and generalization
UniDSeg: Unified Cross-Domain 3D Semantic Segmentation via Visual Foundation Models Prior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UniDSeg, a frozen-VFM encoder with layer-wise learnable visual prompts (depth/low-frequency) and learnable spatial tunability tokens, plus cross-modal 2D-3D learning, for unified domain adaptive and domain generalized 3D semantic segmentation.
Problem
Cross-domain 3D semantic segmentation of LiDAR point clouds under domain shift, covering both unsupervised domain adaptation (target unlabeled) and domain generalization (target unseen), with the goal of leveraging 2D visual foundation models without overwriting their pre-trained knowledge.
Approach
Freeze the VFM (CLIP or SAM ViT) and train only layer-wise lightweight blocks: Modal Transitional Prompting injects sparse-depth and low-frequency-perturbed image embeddings as prompts before each layer, and Learnable Spatial Tunability uses learnable tokens with attention and low-rank adapters after each layer; these 2D features are coupled with a 3D SparseConvNet via xMUDA-style cross-modal KL l
Key results
- On A2D2/sKITTI DA, UniDSeg reaches 57.5% ensemble mIoU, versus 50.0% for the best prior VFMSeg and 40.4% source-only.
- On A2D2/sKITTI DG, UniDSeg reaches 54.4% ensemble mIoU, versus 43.6% for MM2D3D.
- With frozen CLIP ViT-L and only 4.70M trainable encoder parameters, UniDSeg surpasses full fine-tuning by roughly 0.5-2.7% mIoU across scenarios.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add MTP to the frozen VFM (without LST) (ablation) | UniDSeg with CLIP:ViT-B · DG nuScenes:Sing./USA | mIoU (xM): +1.3% | Helped |
| Add MTP to the frozen VFM (without LST) (ablation) | UniDSeg with CLIP:ViT-B · DG A2D2/sKITTI | mIoU (xM): +1.8% | Helped |
| Add LST to the frozen VFM (without MTP) (ablation) | UniDSeg with CLIP:ViT-B · DG nuScenes:Sing./USA | mIoU (xM): +2.1% | Helped |
| Add LST to the frozen VFM (without MTP) (ablation) | UniDSeg with CLIP:ViT-B · DG A2D2/sKITTI | mIoU (xM): +2.4% | Helped |
| Use both MTP and LST instead of only the frozen VFM (ablation) | UniDSeg with CLIP:ViT-B · DG nuScenes:Sing./USA | mIoU (xM) | Helped |
| Use both MTP and LST instead of only the frozen VFM (ablation) | UniDSeg with CLIP:ViT-B · DG A2D2/sKITTI | mIoU (xM) | Helped |
| Increase learnable token length (ablation) | UniDSeg with CLIP:ViT-B · DG nuScenes:USA/Sing. | 2D mIoU | Mixed |
| Increase learnable token length (ablation) | UniDSeg with CLIP:ViT-B · DG A2D2/sKITTI | 2D mIoU | Mixed |
Surprising findings
- Frozen VFMs without specialized design already outperform previous DA3SS methods, and full fine-tuning can beat frozen models in some scenarios.
- Pseudo-label re-training of UniDSeg improves mainly the 3D branch, with almost no gain in 2D prediction.
Limitations
- Pseudo-label re-training yields limited 2D gains, as the limited learnable parameters cannot absorb more target information.
- Evaluated on a restricted set of autonomous-driving scenarios with merged label classes.
- No statistical significance or error bars are reported.
Experimental setup
Datasets: nuScenes A2D2 SemanticKITTI VirtualKITTI
Benchmarks: nuScenes USA/Singapore nuScenes Day/Night VirtualKITTI/SemanticKITTI A2D2/SemanticKITTI A2D2/nuScenes SemanticKITTI validation (fully supervised)
Metrics: mIoU
Baselines: xMUDA MM2D3D VFMSeg SSE BFtD AUDA Dual-Cross DsCML SUMMIT
Builds on: xMUDA visual prompt tuning LoRA SemanticFPN
Closest papers by meaning
- Universal Domain Adaptation for Semantic Segmentation (CVPR 2025 · similarity 0.89)
- EAGLE: Efficient Adaptive Geometry-based Learning in Cross-view Understanding (NeurIPS 2024 · similarity 0.88)
- Adapter Naturally Serves as Decoupler for Cross-Domain Few-Shot Semantic Segmentation (ICML 2025 · similarity 0.88)
- LogoSP: Local-global Grouping of Superpoints for Unsupervised Semantic Segmentation of 3D Point Clouds (CVPR 2025 · similarity 0.87)
- Leveraging Depth and Language for Open-Vocabulary Domain-Generalized Semantic Segmentation (NeurIPS 2025 · similarity 0.87)
- Open-Set Domain Adaptation for Semantic Segmentation (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.