Research map / Image generation / Text-to-image diffusion

DyMO: Training-Free Diffusion Model Alignment with Dynamic Multi-Objective Scheduling

CVPR 2025 · Image generation › Text-to-image diffusion · approach: Inference-time T2I improvement · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DyMO, a training-free plug-and-play method that guides diffusion sampling with a PickScore-style preference reward plus an attention-based semantic alignment loss, dynamically scheduled over denoising stages with a dynamic recurrent (time-travel) strategy.

Problem

Training-free alignment of text-to-image diffusion models to human preferences is underexplored and limited by inaccurate guidance, since preference scores are computed on one-step blurred clean-image estimates, especially in early noisy steps where semantic layout is determined.

Approach

Adds a semantic alignment loss that aligns cross-attention maps of entities and attributes (extracted by an LLM into a positive/negative semantic graph) for early denoising steps, then adaptively shifts weight to the human preference reward in later steps, combined with Polyak step sizes and a dynamic number of recurrent time-travel iterations proportional to gradient norm.

Key results

• Hmm

Measured effects reported in the paper

Surprising findings

• Using open-source Llama-3.3-70B for entity extraction yields 22.58 PickScore, only slightly below GPT-4's 23.07, indicating the LLM component is not strongly dependent on the model choice.

Limitations

• Relies on one-step clean-image approximation, so guidance is underestimated in early steps
• Dynamic time-travel adds substantial inference cost (about 190s per sample in the reported setting)
• Depends on an LLM to extract entities and attributes for the semantic graph

Experimental setup

Datasets: Pick-a-Pic validation set HPSv2 benchmark prompts PartiPrompts

Benchmarks: Pick-a-Pic validation prompts HPSv2 benchmark prompts PartiPrompts

Metrics: PickScore HPSv2 ImageReward Aesthetic Predictor user preference win rate

Baselines: SD V1.5 SDXL DNO PromptOpt FreeDom AlignProp Diffusion-DPO Diffusion-KTO SPO FLUX SD V3.5

Builds on: FreeDom SPO Classifier guidance Polyak step size DPS

Closest papers by meaning

• Diffusion Model Alignment Using Direct Preference Optimization (CVPR 2024 · similarity 0.92)
• Aligning Diffusion Models by Optimizing Human Utility (NeurIPS 2024 · similarity 0.91)
• VideoDPO: Omni-Preference Alignment for Video Diffusion Generation (CVPR 2025 · similarity 0.90)
• InPO: Inversion Preference Optimization with Reparametrized DDIM for Efficient Diffusion Model Alignment (CVPR 2025 · similarity 0.90)
• Calibrated Multi-Preference Optimization for Aligning Diffusion Models (CVPR 2025 · similarity 0.90)
• Improving Long-Text Alignment for Text-to-Image Diffusion Models (ICLR 2025 · similarity 0.89)