Research map / Label-efficient and robust learning / Domain adaptation and generalization
SoMA: Singular Value Decomposed Minor Components Adaptation for Domain Generalizable Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SoMA, a PEFT method that initializes a LoRA-style adapter from the minor singular components of pre-trained ViT weights (keeping the residual frozen), freezes early blocks, and uses annealing weight decay for domain generalizable dense prediction.
Problem
Parameter-efficient fine-tuning of vision foundation models for domain generalization in dense prediction tasks often fails to balance preserving generalizable pre-trained components with learning task-specific features.
Approach
Use SVD of pre-trained weights to place the LoRA adapter on the minor singular components, freezing the principal/residual part so that generalizable top components are preserved; additionally freeze early blocks and anneal weight decay to regain discriminability.
Key results
- SoMA with DINOv2-G reaches 74.07 average mIoU in multi-source synthetic-to-real DGSS, with 6.6M trainable parameters, versus 70.71 for FFT.
- With DINOv2-L on GTAV to Cityscapes/BDD/Mapillary, SoMA achieves 69.31 average mIoU (multi-scale flip) using 4.9M parameters, versus 67.60 for Rein.
- In DGOD with Co-DETR on DINOv2-L, SoMA averages 54.3 mAP@0.5 versus 53.3 for LoRA and 51.0 for FFT, with 4.9M trainable parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SoMA instead of Rein† | DINOv2-L · GTAV → Cityscapes (single-source DGSS) | mIoU: 70.68 -> 73.63 | Helped |
| use SoMA instead of Rein† | DINOv2-L · Cityscapes → BDD (real-to-real DGSS) | mIoU: 66.53 -> 68.08 | Helped |
| use SoMA instead of LoRA | DINOv2-L + Co-DETR · Daytime-Sunny → adverse weather (DGOD average) | mAP@0.5: 53.3 -> 54.3 | Helped |
| freeze early blocks (ablation) | SoMA on DINOv2-L · DGSS average | mIoU: 64.4 -> 65.0 (↑0.6) | Helped |
| freeze early blocks (ablation) | SoMA on DINOv2-L · DGOD average | mAP@0.5: 51.0 -> 51.4 (↑0.4) | Helped |
| tune principal components instead of minor components (ablation) | DINOv2-L adaptation · DGSS average | mIoU: 64.4 -> 66.1 (↑1.1) | Helped |
| tune minor singular components instead of principal components (ablation) | DINOv2-L adaptation · DGSS average | mIoU: 66.1 -> 67.7 (↑2.7) | Helped |
| apply annealing weight decay (ablation) | SoMA on DINOv2-L · DGSS average | mIoU: 67.7 -> 68.3 (↑0.6) | Helped |
| apply annealing weight decay (ablation) | SoMA on DINOv2-L · DGOD average | mAP@0.5: 53.8 -> 54.3 (↑0.5) | Helped |
| apply SoMA to both self-attention and MLP modules rather than only one module type (ablation) | DINOv2-L · GTAV → Cityscapes / BDD / Mapillary (DGSS) | mIoU average: 66.75; 67.35; 68.27 | Helped |
| vary the SoMA rank from 4 to 8 (ablation) | DINOv2-L · DGSS | mIoU average: 66.91 -> 67.71 | Helped |
| vary the SoMA rank from 8 to 16 (ablation) | DINOv2-L · DGSS | mIoU average: 67.71 -> 68.27 | Helped |
Surprising findings
- Tuning minor singular components outperforms tuning principal components by 2.7 mIoU on DGSS in ablation (67.7 vs 66.1 when tuning principal components), contrary to the usual PiSSA-style principal initialization.
- LoRA was found to generalize better than full fine-tuning and several other PEFT methods on DG dense prediction tasks.
Limitations
- Freezing the first eight blocks is a fixed default chosen by heuristic, not adapted per task.
- Gains on the classic ResNet101 DGOD setting are minor.
- Evaluated mainly on driving-scene and detection benchmarks plus one subject-personalization demo; broader domains untested.
Experimental setup
Datasets: GTAV SYNTHIA UrbanSyn Cityscapes BDD100K Mapillary ACDC S-DGOD urban scene detection dataset ImageNet-1k DreamBooth subject images
Benchmarks: GTAV to Cityscapes/BDD100K/Mapillary Cityscapes to BDD100K/Mapillary Cityscapes to ACDC S-DGOD daytime-sunny to night-clear/dusk-rainy/night-rainy/daytime-foggy ImageNet-1k
Metrics: mIoU mAP@0.5
Baselines: Rein FADA SET VPT AdaptFormer SSF LoRA DoRA FFT CLOUDS VLTSeg tqdm
Builds on: LoRA PiSSA Rein Mask2Former Co-DETR
Closest papers by meaning
- Gradient-Guided Annealing for Domain Generalization (CVPR 2025 · similarity 0.87)
- FisherTune: Fisher-Guided Robust Tuning of Vision Foundation Models for Domain Generalized Segmentation (CVPR 2025 · similarity 0.87)
- Stronger Fewer & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentation (CVPR 2024 · similarity 0.87)
- Object-Aware Domain Generalization for Object Detection (AAAI 2024 · similarity 0.86)
- Towards Generalizing to Unseen Domains with Few Labels (CVPR 2024 · similarity 0.86)
- G-NAS: Generalizable Neural Architecture Search for Single Domain Generalization Object Detection (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.