Research map / Image generation / Fast diffusion sampling
Learned Representation-Guided Diffusion Models for Large-Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains latent diffusion models conditioned on self-supervised embeddings for histopathology and satellite patches, then assembles consistent large images via MultiDiffusion-style patch averaging and text-driven auxiliary sampling of SSL grids.
Problem
Diffusion models for large specialized images (histopathology, satellite) need fine-grained conditioning, but patch-level annotations are impractical; generating coherent gigapixel-scale or large images is also hard.
Approach
Use frozen self-supervised embeddings (HIPT, iBOT, CTransPath, DINO) as per-patch conditioning for an LDM, and synthesize large images by assembling a spatial grid of SSL embeddings with MultiDiffusion-style averaging; auxiliary models map text to SSL grids.
Key results
- Patch-level Vanilla FID of 6.98 on TCGA-BRCA 20x, comparable to SoTA PathLDM (7.64 at 10x).
- Augmenting BACH with synthetic large images raises test accuracy from 78% to 83%; NCT-CRC classifier accuracy rises from 93.8% to 96.27% with synthetic data.
- Text-to-large NAIP synthesis reaches CLIP similarity 0.22; expert pathologist agreed with generation labels 89.9% on 100 synthetic CRC images.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add synthetic large-image data to real training data | CLAM-SB · TCGA-BRCA subtyping | AUC: 0.886 -> 0.898 | Helped |
| Add synthetic large-image data to real training data | DSMIL · TCGA-BRCA subtyping | AUC: 0.838 -> 0.856 | Helped |
| Add synthetic large-image data to real training data | CLAM-SB · TCGA-BRCA HRD prediction | AUC: 0.649 -> 0.765 | Helped |
| Add synthetic large-image data to real training data | DSMIL · TCGA-BRCA HRD prediction | AUC: 0.563 -> 0.639 | Helped |
| Augment the training set with generated large images | BACH classifier · BACH test | accuracy: 78 % -> 83 % | Helped |
| Augment the training set with generated patches | ResNet-50 classifier · CRC-VAL-HE validation | accuracy: 93.8 % -> 96.27 % | Helped |
| Use fused diffusion features conditioned on HIPT embeddings instead of HIPT features alone | CLAM-SB · TCGA-BRCA subtyping, 20x, 100% training | AUC: 0.861 -> 0.908 | Helped |
| Use fused diffusion features conditioned on CTransPath embeddings instead of CTransPath features alone | DSMIL · TCGA-BRCA subtyping, 5x, 100% training | AUC: 0.910 -> 0.936 | Helped |
| Generate satellite images from text using the learned embedding pipeline | SSL-guided LDM · NAIP text-to-large-image evaluation | CLIP similarity: 0.22 | Mixed |
Surprising findings
- Diffusion features conditioned on SSL embeddings (LDM features) improve MIL slide classification over SSL features alone in several settings.
Limitations
- Spatial alignment of large images relies on SSL embeddings without explicit topological or low-resolution constraints.
- Text-to-large results are limited by the capability of the VLM used for evaluation; lower performance on BRCA than CRC.
- Large-image augmentation is naive and does not select which regions to augment.
Experimental setup
Datasets: TCGA-BRCA TCGA-CRC Chesapeake Land Cover / NAIP NCT-CRC BACH CRC-VAL-HE-7K
Benchmarks: TCGA-BRCA TCGA-CRC NAIP BACH NCT-CRC CRC-VAL-HE-7K
Metrics: FID CLIP FID embedding similarity AUC accuracy CLIP similarity
Baselines: PathLDM Self-guided diffusion DiffInfinite CLAM-SB DSMIL ConvNeXt V2 SoTA ensemble
Builds on: Latent Diffusion Models MultiDiffusion PathLDM DINO HIPT
Closest papers by meaning
- ZoomLDM: Latent Diffusion Model for Multi-scale Image Generation (CVPR 2025 · similarity 0.90)
- Semantic and Visual Crop-Guided Diffusion Models for Heterogeneous Tissue Synthesis in Histopathology (NeurIPS 2025 · similarity 0.89)
- Dynamic Entity-Masked Graph Diffusion Model for Histopathology Image Representation Learning (AAAI 2025 · similarity 0.88)
- Compositional Discrete Latent Code for High Fidelity, Productive Diffusion Models (NeurIPS 2025 · similarity 0.87)
- Learning Diffusion Models with Flexible Representation Guidance (NeurIPS 2025 · similarity 0.87)
- Nested Diffusion Models Using Hierarchical Latent Priors (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.