Research map / Image generation / Autoregressive and latent generation
Classical generative image synthesis
Synthesize and inpaint images with CNN, GAN, and example-based generators.
147 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Autoregressive and latent generation, keyword cluster “captioning · assessment · synthetic”
State History Benchmarks Tried Baselines Checklist Preprints Who Export
Where it stands
Quiet None of the tracked benchmarks has a report from 2025–26.
35 papers in 2023–24 and 27 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 7.4% to 3.0%.
Open questions the data points to
- Papers report SPADE on ADE20K (mIoU) anywhere from 36.28 to 42: comparisons against it depend on whose number you take.
- Papers report SPADE on DeepFashion (mIoU) anywhere from 76.76 to 87.1: comparisons against it depend on whose number you take.
No longer reported by papers here: Cityscapes (mIoU, last in 2024); Cityscapes (FID, last in 2024); ADE20K (mIoU, last in 2024); ADE20K (FID, last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis (CVPR 2016 · 718 citations)
- 2017 · Semantic Image Inpainting With Deep Generative Models (CVPR 2017 · 1,226 citations)
- 2018 · Generative Image Inpainting With Contextual Attention (CVPR 2018 · 2,538 citations)
- 2019 · Pluralistic Image Completion (CVPR 2019 · 539 citations)
- 2020 · DoveNet: Deep Image Harmonization via Domain Verification (CVPR 2020 · 213 citations)
- 2021 · Pre-Trained Image Processing Transformer (CVPR 2021 · 2,102 citations)
- 2022 · On Aliased Resizing and Surprising Subtleties in GAN Evaluation (CVPR 2022 · 279 citations)
- 2023 · Few-Shot Defect Image Generation via Defect-Aware Feature Manipulation (AAAI 2023 · 109 citations)
- 2024 · Intrinsic Image Diffusion for Indoor Single-view Material Estimation (CVPR 2024 · 43 citations)
- 2025 · Panorama Generation From NFoV Image Done Right (CVPR 2025 · 7 citations)
- 2026 · CondDiff-AMO: Integrating Conditional Diffusion Mechanism for Unified Amodal Mask Generation (AAAI 2026 · 1 citation)
Earliest papers here
- Combining Markov Random Fields and Convolutional Neural Networks for Image Synthesis (CVPR 2016 · 718 citations)
- Generating Images with Perceptual Similarity Metrics based on Deep Networks (NeurIPS 2016 · 160 citations)
- Visualizing and Understanding Deep Texture Representations (CVPR 2016 · 141 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
Cityscapes · mIoU (higher is better)
7 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 68.4 | Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation (CVPR 2020 · 167 citations) |
| 2022 | 69.7 | Retrieval-Based Spatially Adaptive Normalization for Semantic Image Synthesis (CVPR 2022 · 34 citations) |
| 2023 | 72.2 | Edge Guided GANs with Contrastive Learning for Semantic Image Synthesis (ICLR 2023 · 1 citation) |
| 2024 | 76.3 | Unlocking Pre-trained Image Backbones for Semantic Image Synthesis (CVPR 2024 · 9 citations) |
Cityscapes · FID (lower is better)
7 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 49.81 | Semantically Multi-Modal Image Synthesis (CVPR 2020 · 93 citations) |
| 2021 | 38.04 | Diverse Semantic Image Synthesis via Probability Distribution Modeling (CVPR 2021 · 64 citations) |
ADE20K · mIoU (higher is better)
6 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 41.6 | Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation (CVPR 2020 · 167 citations) |
| 2022 | 49.3 | Retrieval-Based Spatially Adaptive Normalization for Semantic Image Synthesis (CVPR 2022 · 34 citations) |
| 2023 | 50.6 | Edge Guided GANs with Contrastive Learning for Semantic Image Synthesis (ICLR 2023 · 1 citation) |
| 2024 | 54.3 | Unlocking Pre-trained Image Backbones for Semantic Image Synthesis (CVPR 2024 · 9 citations) |
ADE20K · FID (lower is better)
6 papers, 2020–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2020 | 31.6 | Local Class-Specific and Global Image-Level Generative Adversarial Networks for Semantic-Guided Scene Generation (CVPR 2020 · 167 citations) |
| 2021 | 29.6 | Diverse Semantic Image Synthesis via Probability Distribution Modeling (CVPR 2021 · 64 citations) |
| 2023 | 25.8 | Edge Guided GANs with Contrastive Learning for Semantic Image Synthesis (ICLR 2023 · 1 citation) |
| 2024 | 22.7 | Unlocking Pre-trained Image Backbones for Semantic Image Synthesis (CVPR 2024 · 9 citations) |
What has been tried
128 papers here report 1,313 measured changes (a component added, removed or swapped, compared with a baseline): helped 71%, no clear effect 5%, hurt 21%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add Stable Diffusion V2 pre-training before fine-tuning on material estimation | InteriorVerse · FID | 113.31 -> 25.42 | Helped | Intrinsic Image Diffusion for Indoor Single-view Material Estimation (CVPR 2024 · 43 citations) |
| Add BKRM to the base LDM | Overall · FID | 96.14 -> 69.80 | Helped | LAKE-RED: Camouflaged Images Generation by Latent Background Knowledge Retrieval-Augmented Diffusion (CVPR 2024 · 16 citations) |
| Add IFR | Calligraphy fonts · FID | 23.7 -> 14.6 | Helped | DeepCalliFont: Few-Shot Chinese Calligraphy Font Synthesis by Integrating Dual-Modality Generative Models (AAAI 2024 · 14 citations) |
| add timestep-wise progressive re-dilation | SD 1.5, 16× 1:1 · FID_r | 94.90 -> 78.22 | Helped | ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models (ICLR 2024 · 14 citations) |
| train with SIN augmentation | ImageNet-D · accuracy (%) | 10.22 -> 8.81 | Hurt | ImageNet-D: Benchmarking Neural Network Robustness on Diffusion Synthetic Object (CVPR 2024 · 13 citations) |
| Remove reconstruction and perceptual losses | CelebA (Bang) · target feature ratio (%) | 0.42 -> 0.08 | Helped | Feature Unlearning for Pre-trained GANs and VAEs (AAAI 2024 · 11 citations) |
| Add FDMM | Flowers · FID | 58.48 -> 38.26 | Helped | Exact Fusion via Feature Distribution Matching for Few-shot Image Generation (CVPR 2024 · 11 citations) |
| Pre-train the discriminator backbone rather than use random initialization | ADE-20K · FID | 52.9 -> 22.7 | Helped | Unlocking Pre-trained Image Backbones for Semantic Image Synthesis (CVPR 2024 · 9 citations) |
| add distortion-map conditioning to the first block only | SUN360 · Distort-FID | 2.68 -> 1.63 | Helped | Panorama Generation From NFoV Image Done Right (CVPR 2025 · 7 citations) |
| Add multi-level synthesisers rather than using a single denoiser | NASA terrain test set · FID | 0.50707 -> 0.44023 | Helped | Terrain Diffusion Network: Climatic-Aware Terrain Generation with Geological Sketch Guidance (AAAI 2024 · 7 citations) |
| Add cross-domain attention | AIM-500 · SAD | 17.8 -> 17.3 | Helped | DRIP: Unleashing Diffusion Priors for Joint Foreground and Alpha Prediction in Image Matting (NeurIPS 2024 · 7 citations) |
| Set noise-sharing K to 200 rather than 0 | Five public datasets (average) · TAR@FAR=1e-6 | 0.4951 -> 0.7180 | Helped | Diff-Palm: Realistic Palmprint Generation with Polynomial Creases and Intra-Class Variation Controllable Diffusion Models (CVPR 2025 · 6 citations) |
Baseline numbers that disagree
The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.
| Baseline | Benchmark | Metric | Reported | Papers |
|---|---|---|---|---|
| SPADE | ADE20K | mIoU | 36.28 – 42 | 2 papers, e.g. CVPR 2021 and CVPR 2020 |
| SPADE | DeepFashion | mIoU | 76.76 – 87.1 | 2 papers, e.g. CVPR 2021 and CVPR 2020 |
| AIM | AIM-500 | SAD | 43.9 – 48.09 | 2 papers, e.g. NeurIPS 2024 and CVPR 2024 |
What reviewers will expect
Datasets and baselines named in the 15 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- ImageNet · 2 of 15 papers
- CelebA-HQ · 2 of 15 papers
Baselines
- IP-Adapter · 2 of 15 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 62 papers; from OpenAlex and the papers' affiliations).
- Alibaba Group · 4 papers
- Tsinghua University · 3 papers
- Zhejiang University · 3 papers
- Shanghai Jiao Tong University · 2 papers
- Harbin Institute of Technology · 2 papers
- Korea Advanced Institute of Science and Technology · 2 papers
- Huazhong University of Science and Technology · 2 papers
- National University of Singapore · 2 papers
- Tencent · 2 papers
- Hong Kong University of Science and Technology · 2 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Retrieval and matching with generative features (140 papers)
- Synthetic training data from generators (111 papers)
- Learning-based image quality assessment (62 papers)
Elsewhere in the topic
- High-resolution diffusion synthesis (140 papers)
- Generative latent space tooling (129 papers)
- Generalizable fake image detection (119 papers)
- Latent diffusion with learned tokenizers (114 papers)
- Compositional scene generation (105 papers)
- Next-scale and token autoregressive images (90 papers)
- Robust diffusion watermarking (79 papers)
- Neural text-to-speech synthesis (72 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).