Research map / Label-efficient and robust learning / Recognition and long-tail
How Well Do Supervised 3D Models Transfer to Medical Imaging Tasks?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a large annotated 3D CT dataset (CT-ORG-style composite of 25 organs plus pseudo tumor labels) and releases supervised pre-trained U-Net, Swin UNETR and SegResNet (SuPreM) that transfer better to 3D segmentation than self-supervised baselines.
Problem
Supervised pre-training for 3D medical imaging is limited by the lack of large annotated 3D volumetric datasets comparable to ImageNet, so transfer learning for 3D segmentation relies on self-supervised or small-data pre-training.
Approach
Construct a large, per-voxel annotated CT dataset from 17 public sources (plus pseudo tumor masks) and use multi-organ segmentation as the supervised pretext task to pre-train standard 3D backbones, then show these supervised features transfer more data- and compute-efficiently than self-supervised ones.
Key results
- SuPreM-pretrained Swin UNETR reaches 91.3% organ segmentation DSC on TotalSegmentator vs 86.4% from scratch, with statistically significant gains across organ, muscle, cardiac and gastro tasks.
- A model pre-trained with 21 CT volumes and 40 GPU hours matches self-supervised pre-training on 5,050 volumes and 1,152 GPU hours (99.6% fewer data, 96.5% less compute).
- Fine-tuning on 512 annotated volumes matches self-supervised models fine-tuned on 1,024 volumes, a 50% annotation reduction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SuPreM supervised pre-training instead of training U-Net from scratch | U-Net · TotalSegmentator v1, organ segmentation | DSC: 88.9{ $\pm$0.6} -> 92.1{ $\pm$0.3} | Helped |
| Use SuPreM supervised pre-training instead of training U-Net from scratch | U-Net · Proprietary dataset, cardiac segmentation | DSC: 38.1{ $\pm$1.1} -> 70.5{ $\pm$0.5} | Helped |
| Increase supervised pre-training data from 21 to 1,575 CT volumes | SuPreM · TotalSegmentator | transfer learning performance: 90.4\% to 91.3\% | Helped |
| Increase self-supervised pre-training data from 1,000 to 5,050 CT volumes | Swin UNETR self-supervised model · TotalSegmentator | transfer learning performance: 89.7\% to 89.9\% | Helped |
| Fine-tune SuPreM on 512 annotated volumes rather than 1,024 | SuPreM · Novel-class segmentation | segmentation performance | No clear effect |
| Use supervised pre-training rather than self-supervised pre-training | SuPreM · TotalSegmentator v1, novel muscle classes | DSC: 93.9{ $\pm$0.1} -> 94.3{ $\pm$0.1} | Helped |
| Use supervised pre-training rather than self-supervised pre-training | SuPreM · TotalSegmentator v1, novel cardiac classes | DSC: 88.9{ $\pm$0.1} -> 90.7{ $\pm$0.1} | Helped |
| Use supervised pre-training rather than self-supervised pre-training | SuPreM · TotalSegmentator v1, novel vertebrae classes | DSC: 86.4{ $\pm$0.3} -> 89.2{ $\pm$0.2} | Helped |
| Use supervised pre-training rather than self-supervised pre-training | SuPreM · Proprietary dataset, novel pancreatic tumor classes | DSC: 43.4{ $\pm$0.3} -> 49.7{ $\pm$0.2} | Helped |
| Apply SuPreM direct inference instead of a dataset-specific model | SuPreM · TotalSegmentator v1 | average DSC: 87.7{ $\pm$0.2} vs 87.6 | No clear effect |
| Apply SuPreM direct inference instead of a dataset-specific model | SuPreM · DAP Atlas | average DSC: 93.3{ $\pm$0.2} vs 93.6 | Hurt |
| Apply SuPreM direct inference instead of a dataset-specific model | SuPreM · Proprietary dataset | average DSC: 86.7{ $\pm$0.2} vs 86.1 | No clear effect |
Surprising findings
- U-Net trained from scratch with the authors' updated MONAI code outperforms older public pre-trained baselines, suggesting prior baseline comparisons were confounded by architecture versions.
- Supervised segmentation pre-training transfers to novel classes (muscles, vertebrae, cardiac, tumors) better than self-supervised pre-training, which the authors attribute to learning general 'objectness'.
Limitations
- Tumor labels are pseudo annotations visually inspected by radiologists without biopsy or pathology confirmation.
- Dataset is CT-only and abdominal-focused; generality to other modalities is not established.
- Some baselines were retrained or reimplemented by the authors and some cited results are not directly comparable.
Experimental setup
Datasets: Pancreas-CT CHAOS CT-ORG BTCV AMOS22 WORD MSD CT tasks LiTS AbdomenCT-1K KiTS FLARE'23 Trauma Detection
Benchmarks: TotalSegmentator v1 proprietary dataset DAP Atlas MSD / external hospital datasets
Metrics: Dice similarity coefficient (DSC) AUC sensitivity specificity
Baselines: Models Genesis UniMiSS Swin UNETR self-supervised (NV) Med3D DoDNet Universal Model learning from scratch
Builds on: ImageNet pre-training Swin UNETR U-Net SegResNet MONAI
Closest papers by meaning
- Knowledge Transfer Scaling Laws for 3D Medical Imaging (NeurIPS 2026 · similarity 0.90)
- Time-to-Event Pretraining for 3D Medical Imaging (ICLR 2025 · similarity 0.87)
- Autoregressive Sequence Modeling for 3D Medical Image Representation (AAAI 2025 · similarity 0.86)
- Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imaging (NeurIPS 2025 · similarity 0.86)
- Do Multiple Instance Learning Models Transfer? (ICML 2025 · similarity 0.86)
- Towards a Comprehensive, Efficient and Promptable Anatomic Structure Segmentation Model Using 3D Whole-Body CT Scans (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.