Research map / Detection and segmentation / Open-set and few-shot detection
Integrating Low-Level Visual Cues for Enhanced Unsupervised Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IL2Vseg, which fuses a spatially-constrained fuzzy clustering branch built on color affinities (low-level cues) with DINO-based high-level features for unsupervised semantic segmentation.
Problem
Self-supervised transformer-based unsupervised semantic segmentation captures high-level semantics but loses local coherence and low-level cues such as color and texture, producing fragmented regions.
Approach
Complement high-level DINO features with a low-level branch of spatially-constrained fuzzy C-means clustering weighted by Gaussian color affinities, fuse the two segmentation vectors with a weighted sum, and optimize with a feature similarity loss plus smoothing and correspondence distillation losses.
Key results
- On COCO-Stuff with ViT-S/8, IL2Vseg reaches 65.1 Acc and 27.6 mIoU, versus 63.2 and 26.7 for SmooSeg.
- On Cityscapes, IL2Vseg reaches 83.0 Acc and 20.6 mIoU, versus 82.8 and 18.4 for EAGLE.
- On MaSTr1325, IL2Vseg reaches 93.6 Acc and 82.5 mIoU, versus 88.3 and 74.1 for SmooSeg.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add low-level visual cues to the self-supervised features | IL2Vseg · COCO-Stuff | Acc.: +16.8 Acc | Helped |
| add low-level visual cues to the self-supervised features | IL2Vseg · COCO-Stuff | mIoU: +3.3 mIoU | Helped |
| add low-level visual cues to the self-supervised features | IL2Vseg · Cityscapes | Acc. | Helped |
| add low-level visual cues to the self-supervised features | IL2Vseg · MaSTr1325 | Acc.: +9.6 ACC | Helped |
| add low-level visual cues to the self-supervised features | IL2Vseg · MaSTr1325 | mIoU: +12 mIoU | Helped |
| add Lfs to Lsmo (ablation) | IL2Vseg · Cityscapes | Acc.: 75.7 -> 81.1 | Helped |
| add Lfs to Lsmo (ablation) | IL2Vseg · Cityscapes | mIoU: 17.4 -> 19.6 | Helped |
| add Lcorr to Lsmo (ablation) | IL2Vseg · Cityscapes | Acc.: 75.7 -> 80.3 | Helped |
| add Lcorr to Lsmo (ablation) | IL2Vseg · Cityscapes | mIoU: 17.4 -> 18.3 | Helped |
| add Lfs and Lcorr to Lsmo (ablation) | IL2Vseg · Cityscapes | Acc.: 75.7 -> 83.0 | Helped |
| add Lfs and Lcorr to Lsmo (ablation) | IL2Vseg · Cityscapes | mIoU: 17.4 -> 20.6 | Helped |
| increase the low-level cue weight to α = 0.5 during training (ablation) | IL2Vseg · segmentation ablation | segmentation performance | Helped |
Surprising findings
- Using only low-level visual cues (alpha=1) gives clearly inferior results, indicating the pre-trained deep features are essential for semantic coherence.
Limitations
- Gains over SmooSeg on COCO-Stuff and Cityscapes are modest (about 0.9 and 0.9 mIoU)
- Hyperparameters (alpha, beta, gamma) were selected on reported ablations, and the fusion weight of 0.5 was tuned on evaluated data
Experimental setup
Datasets: COCO-Stuff Cityscapes Potsdam-3 MaSTr1325
Benchmarks: COCO-Stuff-27 Cityscapes MaSTr1325
Metrics: accuracy mIoU
Baselines: STEGO SmooSeg EAGLE HP TransFGU PiCIE IIC MDC SlotCon MoCoV2
Builds on: STEGO SmooSeg DINO fuzzy c-means
Closest papers by meaning
- Hierarchy-Agnostic Unsupervised Segmentation: Parsing Semantic Image Structure (NeurIPS 2024 · similarity 0.90)
- Unsupervised Semantic Segmentation Through Depth-Guided Feature Correlation and Sampling (CVPR 2024 · similarity 0.88)
- Segment Anything without Supervision (NeurIPS 2024 · similarity 0.88)
- Label-Efficient Few-Shot Semantic Segmentation with Unsupervised Meta-Training (AAAI 2024 · similarity 0.87)
- UNIP: Rethinking Pre-trained Attention Patterns for Infrared Semantic Segmentation (ICLR 2025 · similarity 0.87)
- Progressive Feature Self-Reinforcement for Weakly Supervised Semantic Segmentation (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.