Research map / Image generation / Autoregressive and latent generation
Towards Generalizable Tumor Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffTumor, a three-stage latent diffusion framework (3D VQGAN autoencoder, tumor-mask-conditioned diffusion model, segmentation training on synthetic tumors) that generates realistic early-stage CT tumors generalizing across organs and improves real tumor detection across demographics.
Problem
Creating realistic synthetic tumors that generalize across multiple organs and training tumor detection/segmentation models that transfer to real tumors from different hospitals and patient demographics, without large per-organ annotated tumor datasets.
Approach
Observes that early-stage (<2cm) tumors look similar in CT across liver, pancreas, and kidney, so a latent diffusion model trained on tumors from one organ (even one annotated tumor) can synthesize tumors in other organs conditioned on tumor masks and healthy tissue, which then augment segmentation training.
Key results
- Visual Turing Test: radiologists' specificity on synthetic tumors ranged roughly 22-55%, indicating many synthetic tumors judged real.
- Cross-organ generalization: DiffTumor gives a 10.7% DSC gain over real-tumor training for kidney tumors with nnU-Net.
- On a Hopkins proprietary pancreatic dataset, DiffTumor with U-Net yields average +6.9% DSC and +16.4% sensitivity across demographic groups.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DiffTumor-generated tumors to train instead of real tumors alone | U-Net · liver, all-stage | DSC (%): 62.3±28.3 -> 70.9±21.1 | Helped |
| Use DiffTumor-generated tumors to train instead of real tumors alone | U-Net · pancreas, all-stage | DSC (%): 56.0±24.8 -> 64.8±24.5 | Helped |
| Use DiffTumor-generated tumors to train instead of real tumors alone | U-Net · kidneys, all-stage | DSC (%): 75.1±27.2 -> 84.2±9.5 | Helped |
| Use DiffTumor-generated tumors to train instead of real tumors alone | nnU-Net · liver, all-stage | DSC (%): 64.3±26.5 -> 73.6±18.1 | Helped |
| Use DiffTumor-generated tumors to train instead of real tumors alone | nnU-Net · pancreas, all-stage | DSC (%): 59.9±23.8 -> 63.6±27.7 | Helped |
| Use DiffTumor-generated tumors to train instead of real tumors alone | nnU-Net · kidneys, all-stage | DSC (%): 73.8±20.9 -> 84.5±11.5 | Helped |
| Use DiffTumor-generated tumors to train instead of real tumors alone | SwinUNETR · liver, all-stage | DSC (%): 65.1±23.5 -> 71.4±19.1 | Helped |
| Use DiffTumor-generated tumors to train instead of real tumors alone | SwinUNETR · pancreas, all-stage | DSC (%): 52.2±31.2 -> 62.2±26.1 | Helped |
| Use DiffTumor-generated tumors to train instead of real tumors alone | SwinUNETR · kidneys, all-stage | DSC (%): 80.6±19.6 -> 85.1±8.7 | Helped |
| Train the diffusion model on kidney tumors and synthesize liver tumors | DiffTumor · kidneys→liver, early-stage | tumor-wise sensitivity (%): 74.1 -> 68.8 | Hurt |
| Train the diffusion model on one organ and synthesize tumors in other organs | DiffTumor · liver→pancreas, early-stage | tumor-wise sensitivity (%): 56.3 -> 56.3 | No clear effect |
| Train the diffusion model with timestep T=1 instead of the selected T=4 (ablation) | DiffTumor · tumor synthesis | segmentation performance | Hurt |
Surprising findings
- Training the diffusion model with only one annotated tumor was sufficient to generate useful synthetic tumors, contrary to typical data-hungry generative training.
- Radiologists and SVM/Random Forest/AdaBoost classifiers could not identify the organ of origin of early-stage tumors, near random performance.
Limitations
- Organ-similarity verified only for early-stage tumors (<2cm) and specific tumor subtypes due to public dataset constraints.
- Synthetic tumors are generated with ellipsoid masks refined by radiologists; the approach does not model late-stage tumors.
- Diffusion model does not improve on Hu et al. for kidney-to-liver generalization in one setting; healthy CT subsets were randomly reduced to 120 volumes due to compute limits.
Experimental setup
Datasets: LiTS MSD-Pancreas KiTS Hopkins proprietary pancreatic tumor dataset proprietary healthy CT repository
Benchmarks: LiTS MSD-Pancreas KiTS Hopkins proprietary pancreatic tumor dataset
Metrics: Dice similarity coefficient (DSC) tumor-wise sensitivity specificity accuracy
Baselines: Hu et al. [37] real tumors only training U-Net nnU-Net SwinUNETR
Builds on: Latent Diffusion Models VQGAN Hu et al. tumor synthesis (AI-generated tumors) DDPM
Closest papers by meaning
- Advancing Generalizable Tumor Segmentation with Anomaly-Aware Open-Vocabulary Attention Maps and Frozen Foundation Diffusion Models (CVPR 2025 · similarity 0.84)
- Physics-Regularized Multi-Modal Image Assimilation for Brain Tumor Localization (NeurIPS 2024 · similarity 0.83)
- "Why Is There a Tumor?": Tell Me the Reason, Show Me the Evidence (ICML 2025 · similarity 0.83)
- Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentation (CVPR 2025 · similarity 0.83)
- CAT: Coordinating Anatomical-Textual Prompts for Multi-Organ and Tumor Segmentation (NeurIPS 2024 · similarity 0.83)
- STREAM: Stochastic Riemannian Flow Matching with Anisotropic Decoder for Digital Histopathology Image Generation (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.