Research map / Label-efficient and robust learning / Domain adaptation and generalization
Adapter Naturally Serves as Decoupler for Cross-Domain Few-Shot Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DFN, a residual 1x1-conv adapter that acts as a structure-based domain decoupler for cross-domain few-shot segmentation, plus SAM-SVN to flatten its singular-value loss landscape, reaching 61.98/68.81 average mIoU (ResNet-50) on PASCAL-to-four-target benchmark.
Problem
Cross-domain few-shot semantic segmentation must transfer a model pre-trained on PASCAL to target domains (FSS-1000, Deepglobe, ISIC, Chest X-ray) with only a few labeled samples, facing both domain gap and scarce target data.
Approach
Shows that a residual adapter inserted into deep layers of a frozen backbone naturally absorbs domain-specific information (structure, not loss-based), and builds DFN on this; SAM-SVN applies sharpness-aware minimization only to the singular value matrix of DFN to curb overfitting while preserving domain absorption.
Key results
- DFN with ResNet-50 reaches 61.98% 1-shot and 68.81% 5-shot average mIoU, versus 60.03/65.18 for APM and 61.30/65.09 for APSeg in the reported table (about 2.69 and 4.68 points over the state of the art as claimed).
- ViT-base DFN reaches 63.99% 1-shot and 69.77% 5-shot average mIoU.
- Ablation with ResNet-50: adding DFN raises the 1-shot average from 47.57 to 59.89; adding SAM-SVN further raises it to 61.98.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add DFN to the HSNet baseline (ablation) | HSNet with ResNet-50 · CD-FSS average, 1-shot | MIoU: 47.57 -> 59.89 | Helped |
| add DFN to the HSNet baseline (ablation) | HSNet with ResNet-50 · CD-FSS average, 5-shot | MIoU: 51.38 -> 66.59 | Helped |
| add SAM-SVN to DFN (ablation) | DFN with ResNet-50 · CD-FSS average, 1-shot | MIoU: 60.65 -> 61.98 | Helped |
| add SAM-SVN to DFN (ablation) | DFN with ResNet-50 · CD-FSS average, 5-shot | MIoU: 67.74 -> 68.81 | Helped |
| apply SAM to the singular value matrix rather than encoder, decoder, and DFN collectively (ablation) | DFN with ResNet-50 · CD-FSS average, 1-shot | MIoU: 60.04 -> 61.98 | Helped |
| apply SAM to the singular value matrix rather than encoder, decoder, and DFN collectively (ablation) | DFN with ResNet-50 · CD-FSS average, 5-shot | MIoU: 66.98 -> 68.81 | Helped |
| remove DFN during target-domain fine-tuning (ablation) | DFN with ResNet-50 · CD-FSS average, 1-shot | MIoU: 61.98 -> 56.85 | Hurt |
| reinitialize DFN with Kaiming instead of retaining its trained initialization (ablation) | DFN with ResNet-50 · CD-FSS average, 1-shot | MIoU: 61.98 -> 60.05 | Hurt |
| reinitialize DFN with Xavier instead of retaining its trained initialization (ablation) | DFN with ResNet-50 · CD-FSS average, 1-shot | MIoU: 61.98 -> 59.91 | Hurt |
| reinitialize DFN with Gaussian instead of retaining its trained initialization (ablation) | DFN with ResNet-50 · CD-FSS average, 1-shot | MIoU: 61.98 -> 57.53 | Hurt |
| use SAM-SVN instead of SAM on DFN's full parameters (ablation) | DFN with ResNet-50 · CD-FSS average, 1-shot | MIoU: 60.65 -> 61.98 | Helped |
| use SAM-SVN instead of SAM on DFN's full parameters (ablation) | DFN with ResNet-50 · CD-FSS average, 5-shot | MIoU: 67.74 -> 68.81 | Helped |
Surprising findings
- Adapter decoupling only appears when the adapter is inserted in deep backbone layers with residual connections; loss-based domain decoupling is not needed.
- Removing DFN in the target stage still improves over baseline (56.85 vs 47.57 1-shot average), but full DFN fine-tuning is better.
Limitations
- Does not address many-shot scenarios (stated by authors in the Impact Statement).
- Theoretical analysis relies on strong assumptions (orthogonal projections, approximate linear Jacobians) and is heuristic.
- Some target datasets (e.g., ISIC 1-shot and Deepglobe) still show modest gains against some baselines (for example ISIC 1-shot 36.30 vs APSeg 45.43 in 1-shot average table).
Experimental setup
Datasets: PASCAL-5i FSS-1000 Deepglobe ISIC2018 Chest X-ray
Benchmarks: FSS-1000 Deepglobe ISIC2018 Chest X-ray
Metrics: mIoU CKA similarity MMD loss fluctuation
Baselines: CaNet PANet RPMMs PFENet RePRI HSNet PATNet RestNet PerSAM ABCDFSS APSeg APM
Builds on: HSNet PATNet Sharpness-Aware Minimization SAM-ON BSP
Closest papers by meaning
- Domain-Rectifying Adapter for Cross-Domain Few-Shot Segmentation (CVPR 2024 · similarity 0.93)
- Adapt Before Comparison: A New Perspective on Cross-Domain Few-Shot Segmentation (CVPR 2024 · similarity 0.93)
- Cross-Domain Few-Shot Segmentation via Iterative Support-Query Correspondence Mining (CVPR 2024 · similarity 0.90)
- The Devil is in Low-Level Features for Cross-Domain Few-Shot Segmentation (CVPR 2025 · similarity 0.90)
- APSeg: Auto-Prompt Network for Cross-Domain Few-Shot Semantic Segmentation (CVPR 2024 · similarity 0.89)
- Lightweight Frequency Masker for Cross-Domain Few-Shot Semantic Segmentation (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.