Research map / Detection and segmentation / Weakly supervised detection and segmentation
AllSpark: Reborn Labeled Features from Unlabeled in Transformer for Semi-Supervised Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AllSpark, a channel-wise cross-attention bottleneck with a Semantic Memory and Channel Semantic Grouping that reborns labeled features from unlabeled ones in a pure-transformer pseudo-labeling SSSS framework.
Problem
Separate training flows for labeled and unlabeled data in semi-supervised semantic segmentation let labeled data dominate training, producing low-quality pseudo labels and sub-optimal results.
Approach
An architecture-level bottleneck between encoder and decoder of a ViT segmenter uses channel-wise cross-attention, with labeled features as queries and unlabeled features (stored in a class-wise Semantic Memory filled via similarity-based channel grouping) as keys/values, to reconstruct labeled features so unlabeled data directly intervenes in the labeled flow; at inference it reduces to self-atte
Key results
- On PASCAL VOC 2012 original with 1/8 labels (183), AllSpark reaches 78.41 mIoU vs. 73.10-ish prior SOTA range; ablation shows full AllSpark gives 9.70 mIoU gain over the SemiBaseline at 1/8 split (68.71 to 78.41).
- On PASCAL original 1/16 (92 labels), AllSpark achieves 76.07 mIoU, exceeding UniMatch's 75.2.
- On Cityscapes 1/16 (186 labels), AllSpark achieves 78.33 mIoU vs. UniMatch's 76.6; on COCO 1/512 it achieves 34.10 vs. LogicDiag's 33.07.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AllSpark instead of the strongest listed baseline | PASCAL original · PASCAL original, label fractions 1/16 through Full | mIoU: 76.07 vs 75.2; 78.41 vs 77.19; 79.77 vs 78.8; 80.75 vs 80.33; 82.12 vs 81.77 | Helped |
| Use AllSpark instead of the strongest listed baseline | AllSpark · Cityscapes, label fractions 1/16 through 1/2 | mIoU: 78.33 vs 77.28; 79.24 vs 79.09; 80.56 vs 80.21; 81.39 vs 81.25 | Helped |
| Use AllSpark instead of the strongest listed baseline | AllSpark · PASCAL augmented, label fractions 1/16 through 1/2 | mIoU: 78.32 vs 78.1; 79.98 vs 79.31; 80.42 vs 79.96; 81.14 vs 80.96 | Helped |
| Use AllSpark instead of the strongest listed baseline | AllSpark† · PASCAL augmented †, label fractions 1/16 through 1/2 | mIoU: 81.56 vs 80.94; 82.04 vs 81.92; 80.92 vs 80.62; 81.13 vs 81.01 | Helped |
| Use AllSpark instead of the strongest listed baseline | AllSpark · COCO, label fractions 1/512 through 1/128 | mIoU: 34.10 vs 33.07; 41.65 vs 40.28; 45.48 vs 45.35 | Helped |
| Add channel-wise cross-attention (CCA) to the pseudo-labeling baseline (ablation) | SemiBaseline (SegFormer) · PASCAL 1/8 (183) | mIoU: 68.71 -> 76.77 | Helped |
| Add channel-wise cross-attention (CCA) to the pseudo-labeling baseline (ablation) | SemiBaseline (SegFormer) · PASCAL 1/2 (732) | mIoU: 74.40 -> 78.93 | Helped |
| Add Semantic Memory (S-Mem) to CCA (ablation) | AllSpark · PASCAL 1/8 (183) | mIoU: 76.77 -> 77.62 | Helped |
| Add Semantic Memory (S-Mem) to CCA (ablation) | AllSpark · PASCAL 1/2 (732) | mIoU: 78.93 -> 79.71 | Helped |
| Add Channel Semantic Grouping (CSG) to CCA and S-Mem (ablation) | AllSpark · PASCAL 1/8 (183) | mIoU: 77.62 -> 78.41 | Helped |
| Add Channel Semantic Grouping (CSG) to CCA and S-Mem (ablation) | AllSpark · PASCAL 1/2 (732) | mIoU: 79.71 -> 80.75 | Helped |
| Use AllSpark with ResNet101 and DeepLabV3+ instead of SegFormer-B5 (ablation) | AllSpark · PASCAL original 1/8 (183) | mIoU: 78.41 vs 73.70 | Hurt |
Surprising findings
- AllSpark does not help CNN backbones: with ResNet-101 + DeepLabV3+ it drops to 73.70 mIoU vs. 77.19 for UniMatch, attributed to CNNs' limited receptive fields.
- Previous SOTA methods (CCVC, DGCL, UniMatch) degrade when switched from ResNet-101 to SegFormer-B4/B5 backbones, while AllSpark improves.
Limitations
- Gains are reduced or negative with CNN backbones due to limited receptive fields.
- Requires a class-balanced semantic memory and channel grouping based on probability maps, adding training complexity.
- Tested mainly on SegFormer; generality to other ViT segmenters is not shown.
Experimental setup
Datasets: PASCAL VOC 2012 SBD augmented set Cityscapes COCO
Benchmarks: PASCAL VOC 2012 original PASCAL VOC 2012 augmented Cityscapes COCO
Metrics: mIoU
Baselines: UniMatch CPS U2PL GTA-Seg SemiCVT AugSeg DGCL ESL LogicDiag CFCG CCVC PseudoSeg
Builds on: SegFormer cross-attention pseudo-labeling UniMatch CPS
Closest papers by meaning
- Semi-Supervised Semantic Segmentation via Derivative Label Propagation (AAAI 2026 · similarity 0.89)
- Semi-supervised Semantic Segmentation with Prototype-based Consistency Regularization (NeurIPS 2022 · similarity 0.89)
- Label-Efficient Few-Shot Semantic Segmentation with Unsupervised Meta-Training (AAAI 2024 · similarity 0.88)
- Improving Semi-Supervised Semantic Segmentation with Sliced-Wasserstein Feature Alignment and Uniformity (CVPR 2025 · similarity 0.88)
- SemiDAViL: Semi-supervised Domain Adaptation with Vision-Language Guidance for Semantic Segmentation (CVPR 2025 · similarity 0.88)
- CorrMatch: Label Propagation via Correlation Matching for Semi-Supervised Semantic Segmentation (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.