Research map / Image generation / Autoregressive and latent generation
ZoomLDM: Latent Diffusion Model for Multi-scale Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ZoomLDM, a magnification-conditioned latent diffusion model using SSL embeddings and a summarizer to generate histopathology patches at multiple scales and joint multi-scale large images up to 4096x4096.
Problem
Diffusion models for large-image domains like digital pathology and satellite imagery can only generate small fixed-size patches, lacking global context and conditioning across scales.
Approach
A scale-conditioned latent diffusion model whose conditioning comes from a trainable Summarizer that projects variable-size SSL embedding grids (plus magnification embedding) into a shared cross-magnification latent space, combined with a joint multi-scale sampling procedure based on linear-constraint diffusion inference for coherent large images and super-resolution.
Key results
- ZoomLDM achieves FID 6.77 at 20x vs 6.98 for prior SoTA and 13.42 vs 106.14 at 0.15625x magnification.
- Joint multi-scale sampling yields 1024x1024 CLIP FID 1.23 vs 3.74 (∞-Brush) and generates 4096x4096 images in about 8 minutes per image versus more than 4 hours for prior methods.
- Multi-scale ZoomLDM features (20x + 5x) reach 94.91 AUC on BRCA subtyping and 88.03 AUC on HRD prediction, outperforming UNI and other SSL encoders.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train one shared multi-scale model instead of separate single-magnification models | ZoomLDM · TCGA-BRCA patches at 2.5× to 0.15625× | FID: 20.45 -> 10.73; 106.14 -> 13.42 | Helped |
| use joint multi-scale sampling | ZoomLDM · 1024 × 1024 pathology image generation | CLIP FID: 7.43 -> 1.23 | Helped |
| use joint multi-scale sampling | ZoomLDM · 4096 × 4096 pathology image generation | CLIP FID: 2.75 -> 6.75 | Hurt |
| use joint multi-scale sampling | ZoomLDM · 4096 × 4096 pathology image generation | inference time per image: 4 h -> 8 m | Helped |
| infer embeddings from the low-resolution image instead of using ground-truth embeddings (ablation) | ZoomLDM · TCGA-BRCA 4× super-resolution | LPIPS: 0.250 -> 0.229 | Helped |
| infer embeddings from the low-resolution image instead of using ground-truth embeddings (ablation) | ZoomLDM · BACH 4× super-resolution | LPIPS: 0.245 -> 0.173 | Helped |
| fuse 20× and 5× features instead of using UNI features | ZoomLDM for MIL · BRCA subtyping | AUC: 94.09 -> 94.91 | Helped |
| fuse 20× and 5× features instead of using UNI features | ZoomLDM for MIL · HRD prediction | AUC: 81.79 -> 88.03 | Helped |
Surprising findings
- Single-magnification ZoomLDM 20x features outperform UNI features in MIL despite the model being a generator.
- SSIM and PSNR favor blurry bicubic upsampling over diffusion-based super-resolution outputs.
Limitations
- 4096x4096 FID metrics lag behind prior methods (Crop FID 18.90 vs 11.30 for Graikos et al.).
- Evaluated only on histopathology (plus supplementary satellite results), with the CDM slightly increasing FID compared to real embeddings.
Experimental setup
Datasets: TCGA-BRCA BACH OpenImages NAIP
Benchmarks: TCGA-BRCA BACH BRCA subtyping HRD prediction
Metrics: FID CLIP FID Crop FID SSIM PSNR LPIPS CONCH similarity UNI similarity AUC
Baselines: Graikos et al. [17] DSMIL-style patch diffusion [47] ∞-Brush CompVis LDM ControlNet
Builds on: Latent Diffusion Models UNI Graikos et al. conditional patch diffusion DDIM linear-constraint diffusion inference (DPS-style)
Closest papers by meaning
- Learned Representation-Guided Diffusion Models for Large-Image Generation (CVPR 2024 · similarity 0.90)
- RepLDM: Reprogramming Pretrained Latent Diffusion Models for High-Quality, High-Efficiency, High-Resolution Image Generation (NeurIPS 2025 · similarity 0.90)
- FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion (CVPR 2025 · similarity 0.89)
- ScaleDiff: Higher-Resolution Image Synthesis via Efficient and Model-Agnostic Diffusion (NeurIPS 2025 · similarity 0.89)
- Boosting Generative Image Modeling via Joint Image-Feature Synthesis (NeurIPS 2025 · similarity 0.89)
- DiffuseHigh: Training-Free Progressive High-Resolution Image Synthesis Through Structure Guidance (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.