Research map / 3D vision / 3D generation
Learning General-purpose Biomedical Volume Representations using Randomized Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a 3D UNet contrastively on purely synthetic volumes generated from TotalSegmentator shape templates and randomized appearance models, yielding general-purpose biomedical volume features for registration and few-shot segmentation.
Problem
Volumetric biomedical foundation models generalize poorly because public 3D datasets are small and narrow, and few models support multiple disparate 3D tasks such as registration and segmentation.
Approach
A randomized synthesis data engine composes deformed TotalSegmentator organ templates into label ensembles and renders two appearance-randomized volumes per label map; a multi-positive supervised contrastive loss on multiple decoder layers then teaches appearance-invariant, pose-equivariant voxel features usable both as registration features and as a dataset-agnostic finetuning initialization.
Key results
- ConvexAdam with the proposed features improves median Dice over ConvexAdam by 11 points on L2RAb and 6 on MM-WHS while keeping folds under 0.5%.
- Few-shot segmentation: best average rank across six datasets, with Dice 0.85 on PROMISE12 and 0.80 on FeTA versus 0.80 and 0.78 for random init UNet.
- Only the proposed pretrained features beat the ANTs MutualInfo default in multimodal registration among 3D biomedical foundation models (L2RAb .70 vs .48).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use synthetic biomedical-template pretraining instead of random initialization | UNet · Few-shot segmentation across MSD-Heart, PROMISE12, L2RAb-MRI, FeTA, AMOS-CT, and WUFetal | Dice: .85(.01) -> .89(.01) on MSD-Heart; .80(.02) -> .85(.01) on PROMISE12; .85(.06) -> .86(.06) on L2RAb-MRI; .78(.03) -> .80 | Helped |
| use pretrained features with ANTs instead of its MutualInfo default | ANTs · L2RAbdomenMRCT | Dice: .48(.10) -> .70(.09) | Helped |
| use pretrained features with ANTs instead of its MutualInfo default | ANTs · MM-WHS | Dice: .58(.03) -> .63(.02) | Helped |
| replace the proposed biomedical label generator with alternative label sources (ablation) | synthetic pretraining for registration · L2RAb | Dice: .74(.10) -> .68(.10) (synthetic shapes); .74(.10) -> .57(.10) (Brains); .74(.10) -> .71(.08) (combined) | Hurt |
| replace the proposed biomedical label generator with alternative label sources (ablation) | synthetic pretraining for WUFetal segmentation · WUFetal | Dice: .76(.02) -> .73(.02) (synthetic shapes); .76(.02) -> .74(.02) (Brains); .76(.02) -> .76(.02) (combined) | Mixed |
| set contrastive temperature to 0.07 instead of 0.33 (ablation) | pretraining · L2RAb registration | Dice: .74(.10) -> .58(.10) | Hurt |
| set contrastive temperature to 0.20 instead of 0.33 (ablation) | pretraining · WUFetal and MSD-Heart few-shot segmentation | Dice: .76(.02) -> .78(.02) on WUFetal; .89(.01) -> .91(.01) on MSD-Heart | Helped |
| replace label-supervised contrastive pretraining with denoising (ablation) | pretraining · L2RAb registration | Dice: .74(.10) -> .51(.10) | Hurt |
| remove label supervision from the pretraining loss (ablation) | pretraining · L2RAb registration | Dice: .74(.10) -> .50(.11) | Hurt |
| remove foreground masking from synthetic pretraining (ablation) | pretraining · L2RAb registration and AMOS-CT segmentation | Dice: .74(.10) -> .63(.09) on L2RAb; .61(.01) -> .60(.01) on AMOS-CT | Hurt |
| remove foreground masking and offline augmentations (ablation) | synthetic pretraining · L2RAb registration and AMOS-CT segmentation | Dice: .74(.10) -> .63(.09) on L2RAb; .61(.01) -> .56(.01) on AMOS-CT | Hurt |
| remove foreground masking and all augmentations (ablation) | synthetic pretraining · L2RAb registration and AMOS-CT segmentation | Dice: .74(.10) -> .59(.09) on L2RAb; .61(.01) -> .51(.01) on AMOS-CT | Hurt |
Surprising findings
- Pretraining on real brain label maps (Brains) instead of the synthetic label ensemble degraded abdominal registration Dice to .57, and the proposed model did not beat random initialization on the neurite-oasis adult brain dataset.
Limitations
- Intensity invariance is a suboptimal inductive bias for tasks relying on relative intensities or texture.
- Few-shot segmentation still requires finetuning on each downstream dataset; no promptable or zero-shot segmentation.
- No gain over random initialization on the neurite-oasis adult brain segmentation dataset.
Experimental setup
Datasets: TotalSegmentator L2RAbdomenMRCT MM-WHS MSD-Heart AMOS-CT PROMISE12 FeTA WUFetal SegThy NucMM-M neurite-oasis FreeSurfer brain labels
Benchmarks: L2RAbdomenMRCT MM-WHS MSD-Heart PROMISE12 L2RAb-MRI FeTA AMOS-CT WUFetal SegThy NucMM-M neurite-oasis
Metrics: Dice percentage of folding voxels bootstrapped std
Baselines: ANTs-MI ConvexAdam SynthMorph-shapes uniGradICON Models Genesis SMIT Disruptive AE MedicalNet PrimGeoSeg SuPreM DINOv2
Builds on: supervised contrastive learning Dead Leaves model ConvexAdam Perlin noise SynthMorph
Closest papers by meaning
- Introducing 3D Representation for Dense Volume-to-Volume Translation via Score Fusion (ICML 2025 · similarity 0.88)
- Raptor: Scalable Train-Free Embeddings for 3D Medical Volumes Leveraging Pretrained 2D Foundation Models (ICML 2025 · similarity 0.87)
- Why Invariance is Not Enough for Biomedical Domain Generalization and How to Fix It (NeurIPS 2026 · similarity 0.87)
- Better Tokens for Better 3D: Advancing Vision-Language Modeling in 3D Medical Imaging (NeurIPS 2025 · similarity 0.86)
- VoCo: A Simple-yet-Effective Volume Contrastive Learning Framework for 3D Medical Image Analysis (CVPR 2024 · similarity 0.86)
- Knowledge Transfer Scaling Laws for 3D Medical Imaging (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.