Research map / Detection and segmentation / CNN features
Diffusion-Driven Two-Stage Active Learning for Low-Budget Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a two-stage pixel-level active learning pipeline for low-budget semantic segmentation that combines hierarchical MaxHerding candidate selection with an entropy-augmented diffusion disagreement score (eDALD) over multi-timestep diffusion features.
Problem
Active learning for semantic segmentation under extremely constrained pixel labeling budgets (about one labeled pixel per image), where uncertainty-only methods pick redundant pixels and representation-only methods miss boundary-informative pixels.
Approach
Decouple diversity and uncertainty: first select a diverse candidate pool via local-then-global MaxHerding over diffusion features, then rank candidates by eDALD, which adds a single-sample predictive entropy term to the diffusion-based mutual information (disagreement across noisy feature samples).
Key results
- Two-stage eDALD reaches 37.14 average mIoU across four datasets under the 10-round low-budget regime, versus 34.66 for the best single-stage baseline (Margin with DDPM backbone) and 22.85 for PixelPick with DeepLabV3.
- On CamVid, Herding to eDALD reaches 36.12 mIoU versus 25.14 for eDALD alone, a gain of 10.98 points.
- Switching to Margin after round 10 lets the method reach 90% of fully supervised mIoU in 21-47 rounds while labeling roughly 0.003%-0.007% of pixels.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add MaxHerding before eDALD (ablation) | DDPM active learning · CamVid | mIoU: 25.14 ± 0.57 -> 36.12 ± 0.24; +10.98 | Helped |
| add MaxHerding before entropy sampling (ablation) | DDPM active learning · CamVid | mIoU: 25.26 ± 0.36 -> 30.77 ± 0.44; +5.51 | Helped |
| add MaxHerding before Margin sampling (ablation) | DDPM active learning · CamVid | mIoU: 31.27 ± 1.10 -> 32.77 ± 0.75; +1.50 | Helped |
| add MaxHerding before DALD without entropy augmentation (ablation) | DDPM active learning · CamVid | mIoU: 23.81 ± 3.60 -> 21.05 ± 1.05; -2.76 | Hurt |
| add MaxHerding before BALD (ablation) | DDPM active learning · CamVid | mIoU: 24.59 ± 0.97 -> 22.79 ± 0.89; -1.80 | Hurt |
| add entropy augmentation to DALD after MaxHerding (ablation) | DDPM active learning · CamVid | mIoU: 21.05 ± 1.05 -> 36.12 ± 0.24 | Helped |
| replace MaxHerding with Core-set before eDALD (ablation) | DDPM active learning · ADE-Bed, CamVid, Cityscapes, Pascal-Context | mIoU: 26.70 vs 31.12; 32.95 vs 36.12; 32.20 vs 33.34; 44.58 vs 47.98 | Mixed |
| use MaxHerding alone instead of MaxHerding followed by eDALD (ablation) | DDPM active learning · Pascal-Context | mIoU: 52.04 vs 47.98 | Helped |
| add SAM region expansion from selected pixel prompts (ablation) | 2-Stage eDALD · CamVid | mIoU: 36.10 vs 41.34; +5.24 | Helped |
| add SAM region expansion from selected pixel prompts (ablation) | 2-Stage eDALD · ADE-Bed, Cityscapes, Pascal-Context | mIoU: +0.48 pp, +0.68 pp, and +0.27 pp | Helped |
| use compact blocks {5,8,12,17} instead of original blocks {5,6,7,8,12} (ablation) | DDPM feature extraction · ADE-Bed | mIoU: 46.41 -> 45.58 | Hurt |
| use shallow-only blocks {11–17} instead of compact blocks {5,8,12,17} (ablation) | DDPM feature extraction · ADE-Bed | mIoU: 45.58 -> 22.68 | Hurt |
Surprising findings
- Pure DALD and BALD (disagreement-only) get worse after MaxHerding pre-selection (-2.76 and -1.80 mIoU on CamVid), while their entropy-augmented versions improve substantially.
- On Pascal-Context, MaxHerding alone outperforms its two-stage variant with eDALD (52.04 vs 47.98 mIoU).
Limitations
- DALD relies on sampling multiple noise draws from the diffusion backbone, so it does not transfer to non-diffusion backbones; only the entropy term and MaxHerding stage are backbone-agnostic.
- MaxHerding has quadratic memory and time in the global pool size, which becomes a bottleneck on large datasets (global herding peaks at 38.1 GB on ADE-Bed).
Experimental setup
Datasets: CamVid ADE-Bed Cityscapes Pascal-Context
Benchmarks: CamVid ADE-Bed Cityscapes Pascal-Context
Metrics: mIoU
Baselines: PixelPick BalEntAcq Random Entropy Margin eDALD Core-set eBALD PowerBALD
Builds on: MaxHerding BALD LEDM PixelPick BalEntAcq
Closest papers by meaning
- Pixels to Tokens: Token Space Efficient Active Learning for Low-Budget Semantic Segmentation (NeurIPS 2026 · similarity 0.90)
- Unleashing the Potential of the Diffusion Model in Few-shot Semantic Segmentation (NeurIPS 2024 · similarity 0.89)
- EmerDiff: Emerging Pixel-level Semantic Knowledge in Diffusion Models (ICLR 2024 · similarity 0.89)
- Density Matters: Improved Core-Set for Active Domain Adaptive Segmentation (AAAI 2024 · similarity 0.89)
- Hyperbolic Active Learning for Semantic Segmentation under Domain Shift (ICML 2024 · similarity 0.88)
- Label-Efficient Few-Shot Semantic Segmentation with Unsupervised Meta-Training (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.