Research map / Image generation / Text-to-image diffusion
Generalizing Alignment Paradigm of Text-to-Image Generation with Preferences Through f-Divergence Minimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generalizes the DPO-style text-to-image alignment objective (building on SPO) from reverse KL to f-divergence constraints, analyzes gradient ratios across divergences, and finds Jensen-Shannon gives the best human-value alignment and trade-off with diversity.
Problem
Existing DPO-based text-to-image alignment methods only use the reverse KL divergence constraint, which may reduce generation diversity and leaves other divergences unexplored.
Approach
Derives a generalized alignment loss for text-to-image diffusion models in which the implicit reward is parameterized by f'(density ratio) for any f-divergence, and analyzes how different divergences change the gradient ratio between preferred and dispreferred samples.
Key results
- Jensen-Shannon divergence achieves the best scores on all four human value metrics (e.g., Aesthetics 5.884, ImageReward 0.631, PickScore 21.635, HPS-V2 27.850).
- Reverse KL gives the best image-text alignment (CLIPScore 0.363, VQAScore 0.676), though differences across divergences are small.
- Divergence choice shifts the trade-off between alignment and diversity; JS is judged the best overall trade-off.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace reverse KL divergence with Jensen-Shannon divergence in SPO alignment | SPO with Stable Diffusion v1.5 · HPS-V2 test | CLIPScore: 0.363$\pm$0.049 -> 0.361$\pm$0.049 | No clear effect |
| Replace reverse KL divergence with Jensen-Shannon divergence in SPO alignment | SPO with Stable Diffusion v1.5 · HPS-V2 test | Aesthetics Score: 5.812$\pm$0.514 -> 5.884$\pm$0.514 | Helped |
| Replace reverse KL divergence with Jensen-Shannon divergence in SPO alignment | SPO with Stable Diffusion v1.5 · HPS-V2 test | ImageReward: 0.619$\pm$0.921 -> 0.631$\pm$0.939 | Helped |
| Replace reverse KL divergence with Jensen-Shannon divergence in SPO alignment | SPO with Stable Diffusion v1.5 · HPS-V2 test | PickScore: 21.621$\pm$1.151 -> 21.635$\pm$1.149 | Helped |
| Replace reverse KL divergence with Jensen-Shannon divergence in SPO alignment | SPO with Stable Diffusion v1.5 · HPS-V2 test | HPS-V2: 27.801$\pm$1.352 -> 27.850$\pm$1.388 | Helped |
| Change the divergence constraint used for SPO alignment | SPO with Stable Diffusion v1.5 · HPS-V2 test | generation diversity | Mixed |
| Use alpha-divergence with alpha=0.6 instead of Jensen-Shannon divergence (ablation) | SPO with Stable Diffusion v1.5 · HPS-V2 test | LPIPS: 0.2989 $\pm$ 0.0361 vs 0.3109 $\pm$ 0.0373 | Helped |
| Use alpha-divergence with alpha=0.6 instead of Jensen-Shannon divergence (ablation) | SPO with Stable Diffusion v1.5 · HPS-V2 test | Entropy 1D: 3.9632 $\pm$ 0.1487 vs 3.9275 $\pm$ 0.1991 | Hurt |
Surprising findings
- Reverse KL, the default in prior work, gives the best image-text alignment, while JS wins on human-value metrics.
- Aligned models show lower embedding diversity but higher pixel-level and information-entropy diversity than the original model.
Limitations
- Evaluation limited to a single benchmark model (Stable Diffusion v1.5) and a single benchmark method (SPO) on 400 HPS-v2 test prompts.
- Differences between divergences on image-text alignment are small and not tested for significance beyond mean and std reporting.
- Gradient analysis relies on the assumption that X2/X1 < 1 holds throughout alignment.
Experimental setup
Datasets: HPS-v2 test split
Benchmarks: HPS-v2 test split
Metrics: CLIPScore VQAScore Aesthetics Score ImageReward PickScore HPS-V2 Image-Image CLIP score Entropy 1D Entropy 2D LPIPS
Baselines: Original Model Reverse KL Divergence (SPO) Forward KL Divergence alpha-divergence
Builds on: Direct Preference Optimization Step-aware Preference Optimization f-DPO f-DPG
Closest papers by meaning
- Improving Long-Text Alignment for Text-to-Image Diffusion Models (ICLR 2025 · similarity 0.90)
- CHATS: Combining Human-Aligned Optimization and Test-Time Sampling for Text-to-Image Generation (ICML 2025 · similarity 0.90)
- Discriminative Probing and Tuning for Text-to-Image Generation (CVPR 2024 · similarity 0.89)
- Information Theoretic Text-to-Image Alignment (ICLR 2025 · similarity 0.89)
- Aligning Text to Image in Diffusion Models is Easier Than You Think (NeurIPS 2025 · similarity 0.89)
- SILMM: Self-Improving Large Multimodal Models for Compositional Text-to-Image Generation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.