Research map / Detection and segmentation / Open-set and few-shot detection
LesionLocator: Zero-Shot Universal Tumor Segmentation and Tracking in 3D Whole-Body Imaging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces LesionLocator, a 3D promptable UNet that segments lesions from point/box prompts and tracks them across longitudinal scans via prompt propagation and previous-mask prompting, trained with real and synthetic longitudinal data.
Problem
Zero-shot, promptable 3D lesion segmentation across diverse lesion types and body regions, and lack of longitudinal (multi-timepoint) lesion tracking methods that unify segmentation and retrieval in medical imaging, constrained by scarce longitudinal data.
Approach
A densely promptable residual-encoder 3D UNet (prompts as input channels) combined with a jointly trained deformation-based prompt propagation module and autoregressive use of previous masks as prompts, plus an anatomy-informed synthetic lesion-progression augmentation to create longitudinal training data.
Key results
- Zero-shot box-prompted LesionLocator reaches 83.26 average Dice across six held-out lesion datasets, ~10 points above the next best promptable model (ULS model 74.25 with points; SegVol box 71.26).
- On longitudinal melanoma tracking with 5-fold CV, mask propagation with synthetic data achieves CPM@25 85.96 and Dice@25 79.02 versus 82.50 and 69.27 for Hering et al.
- Synthetic longitudinal data raises point-prompted tracking Dice from 62.62 to 64.33 and reduces MED from 5.09 to 3.89 mm.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LesionLocator with box prompts instead of the strongest reported baseline | Lesion segmentation on six held-out datasets · Six held-out lesion datasets, average | Dice: 74.25 -> 83.26 | Helped |
| Add synthetic longitudinal training data (ablation) | LesionLocator with point prompts · Whole-body melanoma | Dice: 62.62 -> 64.33 | Helped |
| Add synthetic longitudinal training data (ablation) | LesionLocator with point prompts · Whole-body melanoma | CPM@25: 83.62 -> 85.40 | Helped |
| Add synthetic longitudinal training data (ablation) | LesionLocator with point prompts · Whole-body melanoma | MED: 5.09 -> 3.89 | Helped |
| Use box prompts instead of point prompts (ablation) | LesionLocator with synthetic training data · Whole-body melanoma | Dice: 64.33 -> 66.49 | Helped |
| Use box prompts instead of point prompts (ablation) | LesionLocator with synthetic training data · Whole-body melanoma | CPM@25: 85.40 -> 83.95 | No clear effect |
| Propagate the previous generated segmentation as the prompt (ablation) | LesionLocator with synthetic training data · Whole-body melanoma | Dice: 66.49 -> 68.31 | Helped |
| Propagate the previous generated segmentation as the prompt (ablation) | LesionLocator with synthetic training data · Whole-body melanoma | Dice@25: 77.98 -> 79.02 | Helped |
| Use a ball-shaped region rather than a single pixel for point prompts (ablation) | LesionLocator · Lesion segmentation | Dice: 66.56 -> 79.71 | Helped |
| Use a ball-shaped region rather than a single pixel for point prompts (ablation) | LesionLocator · Lesion segmentation | NSD: 59.12 -> 71.61 | Helped |
| Add pretraining (ablation) | LesionLocator with box prompts · Lesion segmentation | Dice: 82.44 -> 83.26 | Helped |
| Add pretraining (ablation) | LesionLocator with box prompts · Lesion segmentation | NSD: 76.37 -> 77.69 | Helped |
Surprising findings
- Point-prompted 2D models performed so poorly they were omitted from the comparison.
- Rendering point prompts as a ball-shaped region rather than a single pixel improved Dice by 13.15 points, attributed to limited receptive field of 3x3x3 kernels.
Limitations
- Demonstrated primarily on CT; other modalities need fine-tuning.
- Closely located or merging lesions remain challenging; re-identification may need an extra click.
- Initial scan requires point prompt; replacing with manual segmentation is future work.
Experimental setup
Datasets: MultiTalent 47-dataset collection lesion-specific public datasets (DeepLesion, etc.) intra-institutional longitudinal melanoma dataset synthetic longitudinal data Coffee-break lung CT colorectal liver metastases CT TCIA liver/adrenal/lymph node datasets
Benchmarks: six held-out zero-shot lesion datasets (colorectal, liver tumor, adrenal tumor, primary liver cancer, lymph node metasta whole-body melanoma longitudinal tracking dataset
Metrics: Dice NSD CPM@25 Dice@25 MED
Baselines: SAM SAM-Med2D MedSAM ScribblePrompt SAM2 SAM-Med3D NVIDIA VISTA SegVol ULS Model Hering et al.
Builds on: SAM MultiTalent ResEnc UNet GradICON anatomy-informed transformations
Closest papers by meaning
- Unified Medical Lesion Segmentation via Self-referring Indicator (CVPR 2025 · similarity 0.84)
- Unleashing the Potential of Vision-Language Pre-Training for 3D Zero-Shot Lesion Segmentation via Mask-Attribute Alignment (ICLR 2025 · similarity 0.83)
- LeFusion: Controllable Pathology Synthesis via Lesion-Focused Diffusion Models (ICLR 2025 · similarity 0.83)
- VISTA3D: A Unified Segmentation Foundation Model For 3D Medical Imaging (CVPR 2025 · similarity 0.82)
- UniPro: Unified Multi-Mode Medical Image Segmentation from 2D Images to 3D Volumes via Propagation (NeurIPS 2026 · similarity 0.82)
- S³-Mamba: Small-Size-Sensitive Mamba for Lesion Segmentation (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.