Research map / Label-efficient and robust learning / Semi-supervised learning
Improving Semi-Supervised Semantic Segmentation with Sliced-Wasserstein Feature Alignment and Uniformity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SWSEG, a semi-supervised semantic segmentation method that adds a Gaussian-approximated Sliced-Wasserstein feature loss (plus decorrelation regularizer) to FixMatch-style consistency training to jointly optimize feature alignment and uniformity.
Problem
Semi-supervised semantic segmentation methods enforce consistency mainly on decoder outputs and ignore feature representations of the backbone encoder; Monte Carlo sliced-Wasserstein estimation of feature alignment/uniformity is costly and nondeterministic.
Approach
Use a closed-form Gaussian-projected sliced-Wasserstein distance between weakly and strongly augmented encoder features, which is claimed to enforce both alignment and uniformity on the hypersphere, combined with a cross-covariance decorrelation term and an MLP projection head.
Key results
- SWSEG reaches 79.0 mIoU on PASCAL VOC 2012 at 1/16 split, outperforming supervised-only (67.2) by 11.8 points.
- On ADE20K at 1/32 split SWSEG achieves 24.4 mIoU vs 15.2 supervised and 22.6 for DT.
- On Cityscapes at 1/16 split SWSEG gets 76.8 mIoU vs 67.9 supervised, but trails DAW/RankMatch at 1/4 (79.5 vs 80.0).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add SWSEG's Gaussian-SWD feature alignment and uniformity loss | SegFormer MiT-B1 · ADE20K 1/32, 1/16, 1/8, 1/4, and 1/2 | mIoU: 24.4 vs 22.6; 26.7 vs 26.0; 30.9 vs 30.6; 34.6 vs 33.7; 38.0 vs 37.3 | Helped |
| add SWSEG's feature alignment and uniformity method | ResNet-101 segmentation model · Cityscapes 1/16, 1/8, and 1/4 | mIoU: 76.8 vs 77.3; 78.6 vs 78.6; 79.5 vs 80.0 | Mixed |
| add SWSEG's feature alignment and uniformity method | ResNet-101 segmentation model · PASCAL VOC 2012 1/16, 1/8, and 1/4 | mIoU: 79.0 vs 78.9; 79.3 vs 79.3; 79.9 vs 80.0 | Mixed |
| add feature alignment using MSE rather than the SWSEG loss (ablation) | SWSEG · PASCAL VOC 2012 | mIoU: 70.2 vs 79.0 | Hurt |
| use cross-entropy feature alignment instead of Gaussian-SWD (ablation) | SWSEG · PASCAL VOC 2012 | mIoU: 77.1 vs 79.0 | Hurt |
| use Monte Carlo SWD with L=128 instead of Gaussian-SWD (ablation) | SWSEG · PASCAL VOC 2012 | mIoU: 77.6 vs 79.0 | Hurt |
| add Gaussian-SWD with Lreg and the MLP projection layer (ablation) | SWSEG · PASCAL VOC 2012 | mIoU: 79.0 | Helped |
| add Lswd (ablation) | SWSEG · PASCAL VOC 2012 1/8, 1/2, and 1 | uniformity loss: -2.5896 -> -2.6592; -2.7095 -> -2.7104; -2.5574 -> -2.6168 | Helped |
Surprising findings
- Monte Carlo SWD with L=128 underperforms the Gaussian-SWD variant and runs out of memory at L=1000, indicating the approximation is both cheaper and better here.
- Plain MSE feature consistency gives lower mIoU (70.2) than no feature loss at all in the ablation.
Limitations
- Gains over recent SOTA are marginal on Cityscapes and PASCAL VOC at larger label fractions (e.g., trails RankMatch and DAW at 1/4 on Cityscapes).
- Uniformity claim relies on a Gaussian approximation whose validity requires weak feature correlation enforced by a heuristic decorrelation term.
Experimental setup
Datasets: PASCAL VOC 2012 Cityscapes ADE20K
Benchmarks: PASCAL VOC 2012 Cityscapes ADE20K
Metrics: mIoU uniformity loss
Baselines: Supervised Only CPS U2PL ReCo GTA-Seg DT UniMatch AugSeg DAW CorrMatch RankMatch DDFP
Builds on: FixMatch Sliced-Wasserstein distance DeepLabV3+ SegFormer Nadjahi et al. Gaussian-projected sliced Wasserstein approximation
Closest papers by meaning
- Semi-Supervised Semantic Segmentation With Cross-Consistency Training (CVPR 2020 · similarity 0.91)
- ScaleMatch: Multi-scale Consistency Enhancement for Semi-supervised Semantic Segmentation (AAAI 2025 · similarity 0.89)
- RWMS: Reliable Weighted Multi-Phase for Semi-supervised Segmentation (AAAI 2024 · similarity 0.88)
- Towards the Uncharted: Density-Descending Feature Perturbation for Semi-supervised Semantic Segmentation (CVPR 2024 · similarity 0.88)
- AllSpark: Reborn Labeled Features from Unlabeled in Transformer for Semi-Supervised Semantic Segmentation (CVPR 2024 · similarity 0.88)
- On the Relaxation of Conditional Independence Assumption for Image Segmentation (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.