Research map / Detection and segmentation / Weakly supervised detection and segmentation
$\epsilon$-Seg: Sparsely Supervised Semantic Segmentation of Microscopy Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ε-Seg, a hierarchical VAE with GMM prior, center-region inpainting, sparse-label contrastive learning and an MLP/Gumbel-softmax segmentation head for sparsely supervised EM and fluorescence segmentation.
Problem
Semantic segmentation of electron microscopy images of biological samples with very few training labels (down to 0.05% or less of image data).
Approach
Combines a hierarchical VAE backbone (from Hierarchical DivNoising) with a Gaussian mixture prior at the top latent level, center-region masked inpainting, and label-based contrastive loss across all latent levels, plus a direct MLP/Gumbel-softmax segmentation head instead of clustering latent embeddings.
Key results
- Outperforms baselines in DSC on the BetaSeg dataset with sparse labels (0.05% or less of image data), per the authors' claim.
- Model is small, with about 3.8M trainable parameters versus 328M for MAESTER.
- Applied also to fluorescence microscopy (Aitslab-bioimaging overlap subset) with competitive results.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use smaller center-mask regions (ablation) | ε-Seg · BetaSeg | DSC | Helped |
| Add entropy loss for unlabeled patches (ablation) | ε-Seg with Gumbel-Softmax · BetaSeg | DSC | No clear effect |
| Use ε-Seg instead of the reported baseline methods | BetaSeg · BetaSeg | DSC | Helped |
Surprising findings
- Smaller masked regions with consistent labels worked best, contrary to the large masking ratios used in transformer-based masked autoencoders.
Limitations
- All experiments are on 2D images; no 3D extension yet.
- Entropy-based loss effectiveness is limited and needs a more adaptive strategy.
- Contrastive loss margin and other hyperparameters require manual grid search tuning.
Experimental setup
Datasets: BetaSeg liver FIBSEM Aitslab-bioimaging1 Aitslab-bioimaging2
Benchmarks: BetaSeg liver FIBSEM Aitslab-bioimaging overlap subset
Metrics: Dice similarity coefficient (DSC/F1)
Baselines: U-Net Labkit MAESTER Vanilla HVAE 4S
Builds on: Hierarchical DivNoising FiLM Gumbel-Softmax U-Net
Closest papers by meaning
- Electron Microscopy Images as Set of Fragments for Mitochondrial Segmentation (AAAI 2024 · similarity 0.86)
- Micro-Bench: A Microscopy Benchmark for Vision-Language Understanding (NeurIPS 2024 · similarity 0.85)
- Weakly Supervised Gland Segmentation with Class Semantic Consistency and Purified Labels Filtration (AAAI 2025 · similarity 0.85)
- Cross-Dimension Affinity Distillation for 3D EM Neuron Segmentation (CVPR 2024 · similarity 0.84)
- Improving Semi-Supervised Semantic Segmentation with Sliced-Wasserstein Feature Alignment and Uniformity (CVPR 2025 · similarity 0.84)
- Alleviate and Mining: Rethinking Unsupervised Domain Adaptation for Mitochondria Segmentation from Pseudo-Label Perspective (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.