Research map / Image generation / Text-to-image diffusion
Personalized and region-controlled generation
Generate subject-driven images with personalization and region control, often without fine-tuning.
136 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2016–2026) · an approach in Text-to-image diffusion, keyword cluster “synthesis · layout · object”
State History Benchmarks Tried Checklist Preprints Who Export
Where it stands
Quiet None of the tracked benchmarks has a report from 2025–26.
40 papers in 2023–24 and 63 in 2025–26 (at ICML, NeurIPS, ICLR, CVPR and AAAI); its share of the topic shrank, from 10.2% to 7.4%.
No longer reported by papers here: CUB (Inception Score, last in 2021); COCO (Inception Score, last in 2021).
How this is decided
A benchmark is tracked when at least four papers of this approach report their own method on the same dataset and metric in at least two different years. Its record is the best number reported up to 2024 compared with the best reported in 2025–26: still improving means at least 1% better, flat means at most 0.2% better with at least two papers from 2025–26 trying. Numbers on a different scale from the rest (a fraction among percentages) and far outliers are left out. Results come from different papers with different setups, so they are signals, not head-to-head comparisons. The share of papers is counted at ICML, NeurIPS, ICLR, CVPR and AAAI.
How it got here
Most cited paper of each year
- 2016 · Learning Deep Structure-Preserving Image-Text Embeddings (CVPR 2016 · 814 citations)
- 2018 · AttnGAN: Fine-Grained Text to Image Generation With Attentional Generative Adversarial Networks (CVPR 2018 · 1,935 citations)
- 2019 · MirrorGAN: Learning Text-To-Image Generation by Redescription (CVPR 2019 · 615 citations)
- 2020 · RiFeGAN: Rich Feature Generation for Text-to-Image Synthesis From Prior Knowledge (CVPR 2020 · 116 citations)
- 2021 · Zero-Shot Text-to-Image Generation (ICML 2021 · 1,121 citations)
- 2022 · CLIP-Forge: Towards Zero-Shot Text-To-Shape Generation (CVPR 2022 · 217 citations)
- 2023 · An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion (ICLR 2023 · 471 citations)
- 2024 · Style Aligned Image Generation via Shared Attention (CVPR 2024 · 131 citations)
- 2025 · CoRe: Context-Regularized Text Embedding Learning for Text-to-Image Personalization (AAAI 2025 · 9 citations)
- 2026 · Implicit Inversion turns CLIP into a Decoder (ICLR 2026 · 1 citation)
Earliest papers here
- Learning Deep Structure-Preserving Image-Text Embeddings (CVPR 2016 · 814 citations)
- Learning to Read Chest X-Rays: Recurrent Neural Cascade Model for Automated Image Annotation (CVPR 2016 · 357 citations)
- AttnGAN: Fine-Grained Text to Image Generation With Attentional Generative Adversarial Networks (CVPR 2018 · 1,935 citations)
Benchmark records
The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.
CUB · Inception Score (higher is better)
5 papers, 2018–2021 · not reported since 2021
| Year | Best so far | Set by |
|---|---|---|
| 2018 | 4.15 | Photographic Text-to-Image Synthesis With a Hierarchically-Nested Adversarial Network (CVPR 2018 · 311 citations) |
| 2019 | 4.62 | Learn, Imagine and Create: Text-to-Image Generation from Prior Knowledge (NeurIPS 2019 · 80 citations) |
| 2021 | 4.91 | TIME: Text and Image Mutual-Translation Adversarial Networks (AAAI 2021 · 40 citations) |
COCO · Inception Score (higher is better)
4 papers, 2018–2021 · not reported since 2021
| Year | Best so far | Set by |
|---|---|---|
| 2018 | 11.86 | Photographic Text-to-Image Synthesis With a Hierarchically-Nested Adversarial Network (CVPR 2018 · 311 citations) |
| 2019 | 26.47 | MirrorGAN: Learning Text-To-Image Generation by Redescription (CVPR 2019 · 615 citations) |
| 2021 | 30.85 | TIME: Text and Image Mutual-Translation Adversarial Networks (AAAI 2021 · 40 citations) |
What has been tried
94 papers here report 1,035 measured changes (a component added, removed or swapped, compared with a baseline): helped 73%, no clear effect 4%, hurt 19%, mixed 3%. Papers mostly publish changes that helped, so "helped" is over-represented.
Ablations in recent papers
| What was changed | Measured on | Result | Effect | Paper |
|---|---|---|---|---|
| Share attention across all images instead of sharing only with the reference image | Style-aligned image sets · style consistency | Helped | Style Aligned Image Generation via Shared Attention (CVPR 2024 · 131 citations) | |
| Remove object masking during training | Person test set · Align | 0.3140 -> 0.3127 | Hurt | InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning (CVPR 2024 · 130 citations) |
| Use dynamic caption templates (TDE) with static captions rather than the two static captions alone | CUHK-PEDES direct transfer · Rank-1 | 46.00 -> 50.32 | Helped | Harnessing the Power of MLLMs for Transferable Text-to-Image Person ReID (CVPR 2024 · 56 citations) |
| add multi-scale visual embeddings | Multi-subject bench · CLIP-T | 0.257 -> 0.302 | Helped | SSR-Encoder: Encoding Selective Subject Representation for Subject-Driven Generation (CVPR 2024 · 41 citations) |
| Use the text prompt instead of removing it | Patchcore ablation · Image-Level (%) | 77.6(±0.6) -> 79.1(±0.6) | Helped | Text-Guided Variational Image Generation for Industrial Anomaly Detection and Segmentation (CVPR 2024 · 24 citations) |
| set selected text proportion to 0% | CTW1500 · Hmean | 78.0 -> 83.29 | Helped | ODM: A Text-Image Further Alignment Pre-training Approach for Scene Text Detection and Spotting (CVPR 2024 · 22 citations) |
| add directional loss LDir | text-driven image editing · text-prompt alignment | Helped | Text-Driven Image Editing via Learnable Regions (CVPR 2024 · 21 citations) | |
| Increase training data from 1M to 6M images | Human image generation test set · FID | 40.08 -> 37.57 | Helped | CosmicMan: A Text-to-Image Foundation Model for Humans (CVPR 2024 · 17 citations) |
| Use AppearNet instead of direct reference-latent concatenation | UCF-101 · FVD | 688.92 -> 508.56 | Helped | MicroCinema: A Divide-and-Conquer Approach for Text-to-Video Generation (CVPR 2024 · 16 citations) |
| set γscope to 0.1 rather than 0.25 | Customization evaluation · CLIP-I | 0.8552 -> 0.8085 | Hurt | RealCustom: Narrowing Real Text Word for Real-Time Open-Domain Text-to-Image Customization (CVPR 2024 · 10 citations) |
| Add Context Embedding Regularization | 24 concepts, 20 prompts · CLIP-T | 0.2504 -> 0.2568 | Helped | CoRe: Context-Regularized Text Embedding Learning for Text-to-Image Personalization (AAAI 2025 · 9 citations) |
| Add proxy-image features and weighted-sum similarity | CIRR test · Recall@10 | 77.25 -> 77.52 | Helped | Imagine and Seek: Improving Composed Image Retrieval with an Imagined Proxy (CVPR 2025 · 9 citations) |
What reviewers will expect
Datasets and baselines named in the 30 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.
Datasets
- OSTF · 2 of 30 papers
- CIRR · 2 of 30 papers
- FashionIQ · 2 of 30 papers
- COCO 2017 · 2 of 30 papers
- Stanford Cars · 2 of 30 papers
- Stanford Dogs · 2 of 30 papers
- Flickr30K · 2 of 30 papers
Baselines
- IP-Adapter · 6 of 30 papers
- BLIP-Diffusion · 4 of 30 papers
- DALL-E 3 · 4 of 30 papers
- SDXL · 4 of 30 papers
- Custom Diffusion · 3 of 30 papers
- Textual Inversion · 3 of 30 papers
- ELITE · 3 of 30 papers
- SegFormer · 2 of 30 papers
Recent preprints
Loading recent arXiv preprints…
Who publishes on it
Institutions on the most papers of this approach since 2023 (of 103 papers; from OpenAlex and the papers' affiliations).
- Google · 6 papers
- Adobe · 4 papers
- Fudan University · 3 papers
- Sun Yat-sen University · 3 papers
- Seoul National University · 3 papers
- University of Oxford · 3 papers
- Carnegie Mellon University · 3 papers
- Shanghai Jiao Tong University · 3 papers
- NVIDIA · 3 papers
- The Hong Kong Polytechnic University · 2 papers
Use it
Download BibTeX of 38 key papersOpen the topic in the interactive map
Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.
Related approaches
Same keyword cluster
- Compositional text-to-image guidance (194 papers)
- Text-driven synthesis and semantic guidance (137 papers)
- Layout-to-image control (164 papers)
- Multi-subject personalization (82 papers)
- Text generation and structured prediction (65 papers)
Elsewhere in the topic
- T2I evaluation and controllable adapters (149 papers)
- Inference-time T2I improvement (117 papers)
- Preference optimization for diffusion (82 papers)
- Concept erasure in diffusion models (76 papers)
- Multi-concept composition (56 papers)
- Fairness and bias in generative models (52 papers)
Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).