Research map / Detection and segmentation / Open-set and few-shot detection

Sparse All-Layer Connector for Domain Generalised Semantic Segmentation

NeurIPS 2026 · Detection and segmentation › Open-set and few-shot detection · approach: Domain-adaptive segmentation

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Sparse All-Layer Connector (SALC), which lets each layer of one frozen foundation model sparsely fuse features from all preceding layers of another model via Sparsegen, plus a frequency-based candidate dropout regulariser, for domain generalised semantic segmentation.

Problem

Multi-model fusion for domain generalised semantic segmentation is restricted to depth-aligned layers, which may not be optimal since foundation models develop different representation hierarchies and domain sensitivities.

Approach

Replace depth-aligned cross-model connectors with a sparse all-layer connector: each query layer adaptively selects a sparse subset of all preceding candidate layers of the other model using Sparsegen with a token-wise learned sparsity factor, and regularises selection with candidate dropout that targets over-used layers.

Key results

• SALC with DINOv2-CLIP reaches 69.40 average mIoU on G→{C,B,M}, surpassing MFuser by 1.58 mIoU.
• On S→{C,B,M}, SALC with DINOv2-CLIP reaches 52.99 average mIoU, +2.65 over MFuser.
• Ablation: fixed or layer-wise sparsity factor drops average mIoU by 1.56 relative to token-wise lambda; uniform dropout reduces average to 67.85 vs 69.40.

Surprising findings

• SALC's gains on SAM-CLIP are smaller and occasionally slightly negative, attributed to weak layer-wise structure of SAM features relative to CLIP.
• Uniform candidate dropout harms generalisation (67.85 avg) relative to no dropout (68.98).

Limitations

• Limited improvement on SAM-CLIP, attributed to weak layer-wise structure of SAM features.
• Smaller gains on the real-to-real C→{B,M} setting.
• Results are reported as averages over a small number of runs for the main comparisons.

Experimental setup

Datasets: GTAV SYNTHIA Cityscapes BDD100K Mapillary

Benchmarks: Cityscapes BDD100K Mapillary

Metrics: mIoU

Baselines: SAN-SAW AdvStyle CMFormer Rein FADA SET SoMA tqdm CLOUDS MFuser DepthForge Vireo

Builds on: Sparsegen Mask2Former MFuser DepthForge

Closest papers by meaning

• Collaborating Foundation Models for Domain Generalized Semantic Segmentation (CVPR 2024 · similarity 0.89)
• Stronger Fewer & Superior: Harnessing Vision Foundation Models for Domain Generalized Semantic Segmentation (CVPR 2024 · similarity 0.89)
• FisherTune: Fisher-Guided Robust Tuning of Vision Foundation Models for Domain Generalized Segmentation (CVPR 2025 · similarity 0.89)
• A Simple Recipe for Language-guided Domain Generalized Segmentation (CVPR 2024 · similarity 0.88)
• MaskViM: Domain Generalized Semantic Segmentation with State Space Models (AAAI 2025 · similarity 0.87)
• Leveraging Depth and Language for Open-Vocabulary Domain-Generalized Semantic Segmentation (NeurIPS 2025 · similarity 0.87)