Research map / Image generation / Autoregressive and latent generation
Compositional scene generation
Generate images from scene graphs and compositional structure.
105 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Autoregressive and latent generation, keyword cluster “motion · layout · object”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Quiet None of the tracked benchmarks has a report from 2025–26.
35 papers in 2023–24 and 36 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 7.4% to 4.0%.
No longer reported by papers here: COCO-Stuff (FID, last in 2024); Visual Genome (FID, last in 2024).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Learning to Generate Posters of Scientific Papers (AAAI 2016 · 16 citations)
- 2017 · Comprehension-Guided Referring Expressions (CVPR 2017 · 148 citations)
- 2018 · Image Generation From Scene Graphs (CVPR 2018 · 854 citations)
- 2019 · StoryGAN: A Sequential Conditional GAN for Story Visualization (CVPR 2019 · 221 citations)
- 2020 · SketchyCOCO: Image Generation From Freehand Scene Sketches (CVPR 2020 · 143 citations)
- 2021 · Spherical Image Generation from a Single Image by Considering Scene Symmetry (AAAI 2021 · 17 citations)
- 2022 · Shadow Generation for Composite Image in Real-World Scenes (AAAI 2022 · 44 citations)
- 2023 · Frido: Feature Pyramid Diffusion for Complex Scene Image Synthesis (AAAI 2023 · 65 citations)
- 2024 · RecDiffusion: Rectangling for Image Stitching with Diffusion Models (CVPR 2024 · 25 citations)
- 2025 · DefectFill: Realistic Defect Generation with Inpainting Diffusion Model for Visual Inspection (CVPR 2025 · 14 citations)
- 2026 · Compositional Generative Modeling from Decentralized Data (ICML 2026 · 0 citations)
Earliest papers here
- Learning to Generate Posters of Scientific Papers (AAAI 2016 · 16 citations)
- Comprehension-Guided Referring Expressions (CVPR 2017 · 148 citations)
- Creativity: Generating Diverse Questions Using Variational Autoencoders (CVPR 2017 · 132 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
COCO-Stuff · FID (lower is better)
6 papers, 2019–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 38.29 | PasteGAN: A Semi-Parametric Method to Generate Image from Scene Graph (NeurIPS 2019 · 60 citations) |
| 2020 | 28.8 | Panoptic-Based Image Synthesis (CVPR 2020 · 46 citations) |
| 2022 | 22.15 | Modeling Image Composition for Complex Scene Generation (CVPR 2022 · 39 citations) |
Visual Genome · FID (lower is better)
4 papers, 2019–2024 · not reported since 2024
| Year | Best so far | Set by |
|---|---|---|
| 2019 | 35.25 | PasteGAN: A Semi-Parametric Method to Generate Image from Scene Graph (NeurIPS 2019 · 60 citations) |
| 2022 | 17.74 | Modeling Image Composition for Complex Scene Generation (CVPR 2022 · 39 citations) |
What has been tried
86 papers here report 873 measured changes (a component added, removed or swapped, compared with a baseline): helped 77%, no clear effect 3%, hurt 18%, mixed 2%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Add stitched-image mask condition | DIR-D test set · SSIM | 0.6125 -> 0.7337 | Helped | RecDiffusion: Rectangling for Image Stitching with Diffusion Models (CVPR 2024 · 25 citations) |
| add point resampling to balance point-cloud density | HoliCity · FID | 131.38 -> 71.98 | Helped | Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion (CVPR 2024 · 21 citations) |
| add heatmap condition cj by concatenating it with image latent z0 | 3DPW · MPJPE | 87.6 -> 75.1 | Helped | DPMesh: Exploiting Diffusion Prior for Occluded Human Mesh Recovery (CVPR 2024 · 17 citations) |
| replace GCN with SGFormer | COCO-Stuff · IS | 16.5 -> 17.5 | Helped | R3CD: Scene Graph to Image Generation with Relation-Aware Compositional Contrastive Control Diffusion (AAAI 2024 · 15 citations) |
| Add DRSE to the baseline | Structured3D test, FOV 90° · FID | 39.450 -> 38.805 | Helped | SphereDiffusion: Spherical Geometry-Aware Distortion Resilient Diffusion Model (AAAI 2024 · 15 citations) |
| Add Low-Fidelity Selection | MVTec AD generated defects · KID | 28.485 → 25.947 | Helped | DefectFill: Realistic Defect Generation with Inpainting Diffusion Model for Visual Inspection (CVPR 2025 · 14 citations) |
| Add weighted noise loss | DESOBAv2 BOS test images · global RMSE | 8.285 -> 7.041 | Helped | Shadow Generation for Composite Image Using Diffusion Model (CVPR 2024 · 14 citations) |
| Add FAG, semantic edges, and spatial edges to the Transformer baseline | UGCrop5K · averaged ACC5 | 54.3 -> 60.8 | Helped | Spatial-Semantic Collaborative Cropping for User Generated Content (AAAI 2024 · 12 citations) |
| add the LDR environment map | Relighting ablation benchmark · PSNR | 25.503 -> 26.052 | Helped | Neural Gaffer: Relighting Any Object via Diffusion (NeurIPS 2024 · 12 citations) |
| Add the tiling constraint | Self-tiling · TS | 0.28 -> 0.03 | Helped | Tiled Diffusion (CVPR 2025 · 9 citations) |
| add PGI | Human-Art · AP (%) | 39.52 -> 47.80 | Helped | GRPose: Learning Graph Relations for Human Image Generation with Pose Priors (AAAI 2025 · 6 citations) |
| add 2D UV layout information to point cloud guidance | Garment texture synthesis · FID | 187.63 -> 96.97 | Helped | TexGarment: Consistent Garment UV Texture Generation via Efficient 3D Structure-Guided Diffusion Transformer (CVPR 2025 · 6 citations) |
What reviewers will expect
Datasets and baselines named in the 21 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- COCO · 2 of 21 papers
Baselines
- LAMA · 2 of 21 papers
- PowerPaint · 2 of 21 papers
- Uni-ControlNet · 2 of 21 papers
- Stable Diffusion · 2 of 21 papers
- DINOSAUR · 2 of 21 papers
- Slot Attention · 2 of 21 papers
- InstructPix2Pix · 2 of 21 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 71 papers; from OpenAlex and the papers' affiliations).
- Peking University · 5 papers
- Shanghai Jiao Tong University · 5 papers
- Google · 4 papers
- South China University of Technology · 3 papers
- NVIDIA · 3 papers
- Beihang University · 3 papers
- Baidu · 3 papers
- Tencent · 3 papers
- Alibaba Group · 3 papers
- Adobe · 3 papers
Use it
Download BibTeX of 40 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Diffusion planning and policies (58 papers)
- Layout generation (39 papers)
Elsewhere in the topic
- Classical generative image synthesis (147 papers)
- High-resolution diffusion synthesis (140 papers)
- Retrieval and matching with generative features (140 papers)
- Generative latent space tooling (129 papers)
- Generalizable fake image detection (119 papers)
- Latent diffusion with learned tokenizers (114 papers)
- Synthetic training data from generators (111 papers)
- Next-scale and token autoregressive images (90 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).