Research map / Label-efficient and robust learning / Recognition and long-tail
Versatile Medical Image Segmentation Learned from Multi-Source Datasets via Model Self-Disambiguation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a single versatile 3D abdominal organ segmentation model from eight partially and sparsely labeled multi-source CT/MRI datasets using ambiguity-aware losses, entropy-minimization regularization, and hierarchical sampling.
Problem
Building a versatile medical image segmentation model normally requires large, diverse, fully annotated data, which is costly; multi-source datasets have only partial or sparse labels, creating label ambiguity and modality, dataset, and class imbalances.
Approach
Model self-disambiguation: ambiguity-aware focal cross-entropy and dice losses compute loss on unannotated voxels only over the possible classes (background plus unannotated structures), combined with a class-agnostic Shannon entropy regularizer and a class→modality→dataset hierarchical sampler, all on a 3D TransUNet that predicts all structures in one forward pass.
Key results
- Overall average DSC of 88.7% on the held-out test set with 3D TransUNet, versus 83.5% for DoDNet and 83.3% for CLIP-driven using the same base network and partial labels.
- Per-structure average DSC of 85.7% for the proposed method, outperforming DoDNet by 5.7 and CLIP-driven by 5.0 points.
- Training on 20% of evenly spaced slices (sparse labels) gave 85.1% to 86.2% DSC, and hybrid partial plus sparse training reached 87.6% to 87.7%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed self-disambiguation approach instead of DoDNet | 3D TransUNet-based segmentation · Test set | DSC: 83.5±15.9 -> 88.7±7.0 | Helped |
| Use the proposed self-disambiguation approach instead of CLIP-driven | 3D TransUNet-based segmentation · Test set | DSC: 83.3±16.3 -> 88.7±7.0 | Helped |
| Train with 8 datasets instead of 3 (ablation) | Ours · Test set | DSC: 83.5±13.9 -> 88.7±7.0 | Helped |
| Add entropy minimization regularization (ablation) | Ours trained on 3 datasets · Test set | DSC: 82.6±15.2 -> 83.5±13.9 | Helped |
| Add entropy minimization regularization (ablation) | Ours trained on 8 datasets · Test set | DSC: 88.2±7.0 -> 88.7±7.0 | Helped |
| Use class→modality→dataset hierarchical sampling instead of random sampling (ablation) | Ours · Test set | DSC: 87.5±7.7 -> 88.7±7.0 | Helped |
| Use class→modality→dataset sampling instead of modality→dataset→class sampling (ablation) | Ours · Test set | DSC: 87.9±7.7 -> 88.7±7.0 | Helped |
| Train with 20% rather than 100% of evenly spaced axial slices (ablation) | Ours · Test set, 8 datasets | DSC: 87.8±8.0 -> 85.1±11.8 | Hurt |
| Train with 20% rather than 100% of evenly spaced sagittal slices (ablation) | Ours · Test set, 8 datasets | DSC: 87.8±7.9 -> 86.2±9.0 | Hurt |
| Train with 20% rather than 100% of evenly spaced coronal slices (ablation) | Ours · Test set, 8 datasets | DSC: 87.8±7.9 -> 86.1±10.3 | Hurt |
| Add 20% sparsely labeled data from five datasets to training with three partially labeled datasets (ablation) | Ours · Test set | DSC: 83.5±13.9 -> 87.6±8.3; 87.7±8.5; 87.6±7.6 | Helped |
Surprising findings
- Sparse slice-level annotations from only 20% of slices achieved DSC comparable to or above the partial-label baselines trained on all slices.
- Adding the entropy minimization regularizer gave a larger gain with fewer training datasets (0.9 points with 3 sets) than with all 8 sets (0.5 points).
Limitations
- Sparse labels are restricted to per-voxel annotations on slices; other weak annotation types such as points or scribbles are not handled.
- Performance is lower on small or variable structures such as adrenal glands and gallbladder, with large standard deviations.
- Evaluation is limited to abdominal structures; generalization to other anatomies is not shown. Urogram is a private dataset.
Experimental setup
Datasets: AbdomenCT-1K AMOS BTCV FLARE 2022 NIH Pancreas TotalSegmentator WORD Urogram
Benchmarks: AbdomenCT-1K AMOS BTCV FLARE 2022 NIH Pancreas TotalSegmentator WORD Urogram
Metrics: Dice Similarity Coefficient (DSC)
Baselines: DoDNet CLIP-driven UNet++ Swin UNETR MedNeXt
Builds on: TransUNet DoDNet CLIP-driven
Closest papers by meaning
- Annotation Ambiguity Aware Semi-Supervised Medical Image Segmentation (CVPR 2025 · similarity 0.87)
- Diversified and Personalized Multi-rater Medical Image Segmentation (CVPR 2024 · similarity 0.87)
- A Semantic Knowledge Complementarity based Decoupling Framework for Semi-supervised Class-imbalanced Medical Image Segmentation (CVPR 2025 · similarity 0.87)
- S2S2: Semantic Stacking for Robust Semantic Segmentation in Medical Imaging (AAAI 2025 · similarity 0.87)
- Constructing and Exploring Intermediate Domains in Mixed Domain Semi-supervised Medical Image Segmentation (CVPR 2024 · similarity 0.86)
- Pancakes: Consistent Multi-Protocol Image Segmentation Across Biomedical Domains (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.