Research map / Image generation / Text-to-image diffusion
IterComp: Iterative Composition-Aware Feedback Learning from Model Gallery for Text-to-Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IterComp, which builds a gallery of six open diffusion models, collects composition-aware preference rankings on attribute binding, spatial and non-spatial relations, trains three reward models, and iteratively fine-tunes SDXL with multi-reward feedback learning.
Problem
Text-to-image diffusion models excel at different aspects of compositional generation (attribute binding, spatial, non-spatial relations), and no single model is strong across all of them.
Approach
Aggregate composition-aware preferences from a gallery of diverse models into per-aspect reward models, then iteratively alternate between fine-tuning reward models and the base diffusion model (ReFL-style), expanding the preference dataset with newly generated images each round.
Key results
- IterComp reaches 0.7982 color, 0.6217 shape, 0.7683 texture on T2I-CompBench, the best among compared methods, and 0.3196 spatial and 0.4873 complex.
- On realism metrics IterComp scores CLIP 0.337, Aesthetic 5.936, ImageReward 1.437, above SDXL (0.322, 5.531, 0.780).
- Combined with RPG, RPG+IterComp improves GenEval overall from 0.660 to 0.723.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IterComp’s composition-aware preference learning and iterative feedback | SDXL · T2I-CompBench | Average Result: 0.4441 -> 0.5554 | Helped |
| Use IterComp’s composition-aware preference learning and iterative feedback | SDXL · T2I-CompBench | CLIP Score: 0.322 -> 0.337 | Helped |
| Use IterComp’s composition-aware preference learning and iterative feedback | SDXL · Image realism evaluation | Aesthetic Score: 5.531 -> 5.936 | Helped |
| Use IterComp’s composition-aware preference learning and iterative feedback | SDXL · Image realism evaluation | ImageReward: 0.780 -> 1.437 | Helped |
| Use IterComp for generation | SDXL · Single image generation | Inference Time: 5.63 s/Img -> 5.63 s/Img | No clear effect |
| Use IterComp as the RPG backbone | RPG · DPG-Bench | Average: 81.28 -> 84.72 | Helped |
| Use IterComp as the RPG backbone | RPG · Genval | Overall: 0.660 -> 0.723 | Helped |
| Increase the model gallery size (ablation) | IterComp · Compositional generation evaluation tasks | performance | Helped |
| Add composition-aware iterative feedback learning (ablation) | IterComp · Compositional generation | accuracy | Helped |
| Add composition-aware iterative feedback learning (ablation) | IterComp · Image generation | Aesthetic Score | Helped |
| Increase the number of feedback-learning iterations (ablation) | IterComp · Compositional generation | model preference convergence | Helped |
Surprising findings
- Inference time of IterComp (5.63 s/img) matches SDXL, while FLUX-dev takes 23.02 s/img in the authors' measurement.
Limitations
- Preference labels come from only three expert annotators, and the reward models and dataset are small (3,500 prompts).
- Iterative loop relies on the reward models' own judgments, so gains may be bounded by reward-model accuracy.
- Layout-based and LLM-based baselines are evaluated with GPT-4o-derived layouts, which may affect fair comparison.
Experimental setup
Datasets: T2I-CompBench DiffusionDB
Benchmarks: T2I-CompBench DPG-Bench GenEval
Metrics: attribute binding score (color, shape, texture) spatial score non-spatial score complex score CLIP score Aesthetic score ImageReward inference time user study win rate
Baselines: Stable Diffusion XL FLUX RPG InstanceDiffusion GenTron PixArt-alpha Diffusion-DPO ImageReward LMD+ GLIGEN
Builds on: Reward Feedback Learning (ReFL) Diffusion-DPO ImageReward Direct Preference Optimization
Closest papers by meaning
- Improving Compositional Generation with Diffusion Models Using Lift Scores (ICML 2025 · similarity 0.90)
- Role Bias in Diffusion Models: Diagnosing and Mitigating through Intermediate Decomposition (NeurIPS 2025 · similarity 0.89)
- RealCompo: Balancing Realism and Compositionality Improves Text-to-Image Diffusion Models (NeurIPS 2024 · similarity 0.89)
- Progressive Compositionality in Text-to-Image Generative Models (ICLR 2025 · similarity 0.89)
- Compositional Discrete Latent Code for High Fidelity, Productive Diffusion Models (NeurIPS 2025 · similarity 0.89)
- ORCA: Hunting Compositional Failures in Text-to-Image Diffusion (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.