Research map / Image generation / Text-to-image diffusion
Policy Optimized Text-to-Image Pipeline Design
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains an LLM (Llama3.1-8B) to generate ComfyUI workflows via SFT on unscored flows, then GRPO guided by an uncertainty-aware ensemble of ModernBERT surrogate reward models, plus a CFG-style extrapolation between SFT and GRPO models.
Problem
Automatically designing prompt-adaptive text-to-image ComfyUI workflows is expensive when training requires generating and scoring images for each flow, and prior LLM-based methods (ComfyGen) mostly copy training flows rather than generating novel ones.
Approach
Replace image generation during RL with an ensemble of ModernBERT surrogate reward models predicting HPS-style scores from prompt-flow text (with prefix-based per-component rewards and uncertainty-based zeroing), combine with GRPO on a compact workflow encoding, and apply classifier-free-guidance-style logit extrapolation between SFT and GRPO models.
Key results
- FlowRL achieves GenEval overall 0.61, matching ComfyGen (0.61) and beating GenArtist (0.47) and SDXL variants (0.55-0.59).
- FlowRL wins 60% HPSv2 head-to-head vs ComfyGen without CFG, 63% at gamma=2 in originality table, with 0% of generated flows existing in training data vs ComfyGen's 94%.
- Removing the SFT stage yields 0% win rate vs ComfyGen and vs the full model; ComfyGen with the new encoding reaches only 37% win rate vs original ComfyGen.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FlowRL instead of ComfyGen | Text-to-image workflow generation · GenEval overall | Overall score: 0.61 vs 0.61 | No clear effect |
| Use FlowRL instead of ComfyGen | Text-to-image workflow generation · GenEval | Two object: 0.82 -> 0.85 | Helped |
| Use FlowRL instead of ComfyGen | Text-to-image workflow generation · GenEval | Attribute binding: 0.29 -> 0.38 | Helped |
| Add dual-model guidance with γ=1.5 (ablation) | FlowRL · CivitAI prompt set | HPSv2 win rate vs ComfyGen: 59% -> 60% | Helped |
| Increase CFG strength from γ=1.5 to γ=2 (ablation) | FlowRL · CivitAI prompt set | HPSv2 win rate vs ComfyGen: 60% -> 63% | Helped |
| Increase CFG strength from γ=1.5 to γ=2 (ablation) | FlowRL · CivitAI prompt set | Unique ratio: 41% -> 8% | Hurt |
| Add prefix reward (ablation) | FlowRL reward model · CivitAI prompt set | HPSv2 win ratio vs full model: 42 | Helped |
| Add the uncertainty reward cutoff (ablation) | FlowRL reward model · CivitAI prompt set | HPSv2 win ratio vs full model: 45 | Helped |
| Use a 7-model reward ensemble instead of 1, 3, or 5 models (ablation) | FlowRL reward model · CivitAI prompt set | HPSv2 win ratio vs full model: 33; 34; 36 | Helped |
| Add GRPO tuning after SFT (ablation) | FlowRL · CivitAI prompt set | HPSv2 win ratio vs full model: 19 | Helped |
| Add the SFT initialization stage before GRPO (ablation) | FlowRL · CivitAI prompt set | HPSv2 win ratio vs ComfyGen: 0 | Helped |
Surprising findings
- Stronger CFG guidance (gamma=2) raised HPSv2 win rate but dropped flow uniqueness from 41% to 8%, a diversity-quality trade-off.
- Reward model generalization drops sharply for flows with unseen components (Pearson 0.152), though it remains 0.928 for parameter-only changes.
Limitations
- Only supports text-to-image workflows, with no editing or video modules.
- Adding new workflow components requires retraining the entire stack.
- Reward model generalizes poorly to flows with entirely unseen components.
Experimental setup
Datasets: ComfyGen flow and prompt dataset CivitAI prompts
Benchmarks: GenEval CivitAI test prompts (HPSv2)
Metrics: GenEval overall score HPS v2 win rate unique flow ratio NLD ratio exists-in-data rate user study win rate
Baselines: SDXL JuggernautXL DreamShaperXL DPO-SDXL GenArtist RPG-DiffusionMaster ComfyGen
Builds on: ComfyGen GRPO classifier-free guidance ModernBERT
Closest papers by meaning
- Learning to Sample Effective and Diverse Prompts for Text-to-Image Generation (CVPR 2025 · similarity 0.89)
- Sketch Then Paint: Hierarchical Reinforcement Learning for Diffusion Multi-Modal Large Language Models (NeurIPS 2026 · similarity 0.88)
- Elucidating the design space of language models for image generation (ICML 2025 · similarity 0.87)
- Rich Human Feedback for Text-to-Image Generation (CVPR 2024 · similarity 0.87)
- T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT (NeurIPS 2025 · similarity 0.87)
- EvolveDirector: Approaching Advanced Text-to-Image Generation with Large Vision-Language Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.