Research map / Image generation / Text-to-image diffusion
Preference Adaptive and Sequential Text-to-Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds PASTA, a value-based RL agent that selects prompt-expansion slates for multi-turn interactive text-to-image generation, trained on a new human-rater sequential dataset plus EM-learned user simulator data.
Problem
Single-turn text-to-image generation often fails to capture a user's evolving or underspecified intent; interactive T2I needs multi-turn adaptation to user preferences without a sequential preference dataset.
Approach
Treat iterative prompt expansion as a latent-user MDP: an LMM generates a large candidate set of prompt expansions, and a value-based selector (IQL-trained, slate value decomposed as an average of per-prompt values) picks a diverse slate each turn; an EM-trained user-type choice/utility simulator generates extra training trajectories.
Key results
- PASTA receives an 85% relative improvement over the baseline Gemini 1.5 Flash when raters compare final-turn images.
- PASTA achieves a 20% relative improvement over the baseline when transferred to Flux.1 without retraining.
- Training on real human data improves over the pre-trained LMM, while synthetic-only data slightly declines performance; combined data performs best.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PASTA instead of the Gemini 1.5 Flash baseline for final-turn image generation | Gemini 1.5 Flash baseline · Human raters, final-turn comparison | relative improvement: 85% | Helped |
| Use PASTA with Flux.1 without retraining on Flux.1 data | Gemini 1.5 Flash baseline · Human raters, Flux.1 final-image comparison | relative improvement: 20% | Helped |
| Train the value function on real human-rater data (ablation) | PASTA · Human-rater evaluation | turn-over-turn preference | Helped |
| Train the value function on simulated user data alone (ablation) | PASTA · Human-rater evaluation | turn-over-turn preference | Hurt |
| Train the value function on both real human-rater and simulated user data (ablation) | PASTA · Human-rater evaluation | turn-over-turn preference | Helped |
| Use PASTA for sequential image refinement | PASTA · Simulated-user rollouts | user-model score | Helped |
Surprising findings
- Training with synthetic simulator data alone leads to a slight decline in performance relative to the baseline LMM.
Limitations
- Possible rater-group bias because experiment arms could not be run concurrently and rater pool may not be representative.
- User simulator uses simplified assumptions (choice equals preference) and may not reflect real user behavior.
- Human evaluation uses a small set of initial prompts and a single rater pool.
Experimental setup
Datasets: PASTA human rater sequential data HPS v2 Pick-a-Pic Simulacra Aesthetic Captions
Benchmarks: Pick-a-Pic test set HPS test set PASTA human rater dataset
Metrics: human rater better/worse/same preference Spearman rank correlation choice accuracy cross-turn preference accuracy
Baselines: Gemini 1.5 Flash (untrained)
Builds on: Implicit Q-Learning CLIP Bradley-Terry preference model latent MDP
Closest papers by meaning
- Multimodal Large Language Models Make Text-to-Image Generative Models Align Better (NeurIPS 2024 · similarity 0.89)
- Rich Human Feedback for Text-to-Image Generation (CVPR 2024 · similarity 0.89)
- Subject-driven Text-to-Image Generation via Preference-based Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
- Boosting Text-to-Video Generative Model with MLLMs Feedback (NeurIPS 2024 · similarity 0.88)
- AGFSync: Leveraging AI-Generated Feedback for Preference Optimization in Text-to-Image Generation (AAAI 2025 · similarity 0.88)
- Learning to Sample Effective and Diverse Prompts for Text-to-Image Generation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.