Research map / Image generation / Text-to-image diffusion

Training-free Dense-Aligned Diffusion Guidance for Modular Conditional Image Synthesis

CVPR 2025 · Image generation › Text-to-image diffusion · approach: Layout-to-image control · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes training-free, plug-and-play dense alignment guidance modules (DCA, DGA, DMA) that let diffusion models combine text, layout, and drag conditions for modular conditional image synthesis.

Problem

Existing conditional diffusion methods are task-specific and handle only one condition form, and combining text, layout, and drag conditions without omission is non-trivial due to concept mismatching, geometry inconsistency, and motion disharmony.

Approach

Treats conditional synthesis as modular combination of text, layout, and drag condition units, each enforced by a training-free guidance module computed on the one-step predicted clean image: DCA aligns attribute/relation concepts with CLIP region embeddings, DGA enforces IoU, size-ratio, and centroid-distance constraints on segmentation masks, and DMA aligns dense drag flow with RAFT optical flow

Key results

• DCA on Stable Diffusion v3.0 improves COCO long-description FID from 10.51 to 9.22 and CLIP score from 0.2861 to 0.2946.
• DGA on A&R raises mask-to-image IoU from 38.97 to 49.52 (27.1% gain) and SOA-I from 78.80 to 84.75; adding DCA reaches IoU 50.61 and SOA-I 86.33.
• On DragBench, DMA combined with DCA achieves better Mean Distance and Image Fidelity than DragDiffusion; DMA alone slightly lowers image fidelity.

Measured effects reported in the paper

Surprising findings

• DMA alone slightly reduces image fidelity compared with DragDiffusion, attributed to over-smoothing from the motion field, and DCA partly mitigates this.

Limitations

• Relies on the capability of the foundation model; counterfactual scenes (e.g., upward-flowing river) are hard to generate.
• Layout and motion guidance depend on external detection, segmentation, and optical flow models whose errors propagate.
• Guidance applied only during the first 70% of timesteps, with hand-tuned weighting schedules and loss coefficients.

Experimental setup

Datasets: COCO DenseDiffusion benchmark DrawBench DragBench

Benchmarks: COCO DenseDiffusion DrawBench DragBench

Metrics: FID CLIP score IoU SOA-I Image Fidelity (1-LPIPS) Mean Distance

Baselines: ParaDiffusion PIXART-α SDXL ELLA PIXART-Σ Composable Diffusion Structure Diffusion SD-Pww Dense Diffusion A&R DragDiffusion

Builds on: DDIM DenseDiffusion DragDiffusion DOODL A&R

Closest papers by meaning

• Referee Can Play: An Alternative Approach to Conditional Generation via Model Inversion (ICML 2024 · similarity 0.88)
• Toward Guidance-Free AR Visual Generation via Condition Contrastive Alignment (ICLR 2025 · similarity 0.87)
• Compose and Conquer: Diffusion-Based 3D Depth Aware Composable Image Synthesis (ICLR 2024 · similarity 0.87)
• Control and Realism: Best of Both Worlds in Layout-to-Image without Training (ICML 2025 · similarity 0.86)
• Diff-Instruct with Diffused Reward: Towards Principled One-step Generator RL (NeurIPS 2026 · similarity 0.86)
• A Simple Approach to Unifying Diffusion-based Conditional Generation (ICLR 2025 · similarity 0.86)