Research map / Image generation / Text-to-image diffusion

Where to Look Is Not How to Fix: Pre-Denoising Diagnostics and Modality-Dependent Control in Diffusion Composition

NeurIPS 2026 · Image generation › Text-to-image diffusion · approach: Compositional text-to-image guidance · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows via text-only Compositional Stress Index and per-block cross-attention interventions in SD1.5/SDXL that compositional stress is diagnosable pre-denoising but diagnostic sites and intervention-responsive sites dissociate.

Problem

Text-to-image diffusion models fail on rare attribute-object compositions; it is unclear whether the defect arises during denoising or is already present in text conditioning, and whether the site most diagnostic of failure is also the site most effective for intervention.

Approach

A text-only non-factorization residual (Compositional Stress Index) that tests whether full-prompt text embeddings can be reconstructed from attribute, object, and context component embeddings, combined with a per-block comparison of diagnostic accessibility (attention-based anchor-vs-stress binding contrast) against intervention response across three cross-attention modalities (boost, selective z

Key results

• CSI separates anchor from stress prompts with ROC-AUC 1.0 across SD1.5, SDXL, and SD3 text path; a frequency-only baseline reaches 0.79.
• Downstream cross-attention intervention increases CHR by +0.0272 (95% CI [0.0092, 0.0469]) on a six-prompt localization subset, while token-level embedding repair decreases CHR by -0.0680.
• Broad zeroing of SDXL U0 cross-attention reduces CHR by -0.1338 (q=0.0010), while D2 has the largest positive signed DA mean; boost CE effects are small (+0.006 to +0.008) and do not survive per-block FDR.

Surprising findings

• Residual-minimizing embedding adapters reduce representation residuals substantially but do not improve color hit rate (observed delta CHR of 0 on seven prompts; token adapter reduces CHR on localization subset).
• The deep encoder D2 has the largest positive diagnostic accessibility yet boost at D2 is no better than baseline and is most destructive in early denoising.

Limitations

• Boost effects are small and do not survive within-model FDR correction across blocks; CE is color-only on 52 valid prompts per seed.
• CSI separation at ROC-AUC 1.0 partly reflects the controlled protocol, where stress prompts are deliberately rare; the combined CSI includes internal attention features so only the text component is strictly pre-denoising.
• Evaluation limited to SD1.5 and SDXL UNets for denoiser interventions; SD3 used only for text-path diagnostics; no DiT backbone tested.

Experimental setup

Datasets: T2I-CompBench CLEVR ARO

Benchmarks: 432-prompt anchor-stress dataset (216 pairs)

Metrics: ROC-AUC color hit rate (CHR) color effect (CE) diagnostic accessibility (DA) representation residual Spearman correlation

Baselines: Attend-and-Excite SynGen StructureDiffusion Divide-and-Bind WiCLP

Builds on: Attend-and-Excite SynGen WiCLP Prompt-to-Prompt FreeU

Closest papers by meaning

• FreeControl: Efficient, Training-Free Structural Control via One-Step Attention Extraction (NeurIPS 2025 · similarity 0.88)
• Interpretable Diffusion via Information Decomposition (ICLR 2024 · similarity 0.88)
• Unsupervised Region-Based Image Editing of Denoising Diffusion Models (AAAI 2025 · similarity 0.88)
• Denoising Task Difficulty-based Curriculum for Training Diffusion Models (ICLR 2025 · similarity 0.88)
• Why DiT Models Underperform as Representation Learners without Long Skip Connections (NeurIPS 2026 · similarity 0.88)
• ORCA: Hunting Compositional Failures in Text-to-Image Diffusion (NeurIPS 2026 · similarity 0.87)