Research map / Image generation / Fast diffusion sampling
Normalized Attention Guidance: Universal Negative Guidance for Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Normalized Attention Guidance (NAG), a training-free method that extrapolates cross-attention features between positive and negative prompts, normalizes them with L1 clipping and blends them back, to restore negative guidance in few-step and multi-step diffusion image and video models.
Problem
Negative guidance (suppressing unwanted attributes) fails in few-step diffusion models because CFG's output-space extrapolation breaks when positive and negative branches diverge; prior attention-space method NASA is unstable, especially on DiT architectures.
Approach
Apply extrapolation directly in cross-attention output space (Z+ + phi(Z+ - Z-)), then constrain it with a per-token L1-norm ratio clipping and refine by blending with the positive features, making attention-space negative guidance stable across UNet and DiT models.
Key results
- On 4-step Flux-Schnell, NAG raises CLIP from 31.4 to 32.0 and ImageReward from 1.029 to 1.099 while lowering FID from 25.47 to 24.46.
- On 1-step NitroSD-Realism, NAG lowers FID from 26.21 to 23.98 and raises ImageReward from 0.847 to 0.948.
- User study shows +25.0% text preference and +33.9% visual preference for NAG on Flux-Schnell.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add NAG to the baseline sampler | Flux-Schnell · COCO-5K, 4 steps | CLIP Score: 31.4 -> 32.0 | Helped |
| Add NAG to the baseline sampler | Flux-Schnell · COCO-5K, 4 steps | FID: 25.47 -> 24.46 | Helped |
| Add NAG to the baseline sampler | Flux-Schnell · COCO-5K, 4 steps | ImageReward: 1.029 -> 1.099 | Helped |
| Add NAG alongside CFG | SD3.5-Large · COCO-5K, 25 steps, CFG | CLIP Score: 31.8 -> 32.0 | Helped |
| Add NAG alongside CFG | SD3.5-Large · COCO-5K, 25 steps, CFG | FID: 25.07 -> 25.42 | Hurt |
| Add NAG alongside CFG and PAG | SD3.5-Large · COCO-5K, 25 steps, CFG + PAG | PFID: 37.93 -> 39.09 | Hurt |
| Add NAG to CFG | SDXL · COCO-5K, 25 steps | FID: 23.25 -> 20.90 | Helped |
| Add NAG to CFG + PAG | SDXL · COCO-5K, 25 steps | ImageReward: 0.748 -> 0.914 | Helped |
| Add NAG rather than NASA | NitroSD-Realism · COCO-5K, 1 step | FID: 25.86 -> 23.98 | Helped |
| Remove both normalization and refinement (ablation) | NAG · Few-step sampling at guidance scales above 5 | FID | Hurt |
| Remove refinement (ablation) | NAG · Few-step sampling at guidance scale 20 | FID | Hurt |
| Remove normalization (ablation) | NAG · Few-step sampling at high guidance scales | FID | Hurt |
Surprising findings
- NASA fails completely on DiT architectures, producing unstable outputs that motivated the normalization design.
- On SD3.5-Large with CFG+PAG, NAG increases PFID by 1.16 while improving CLIP and ImageReward.
Limitations
- Excessive guidance scales or poorly formulated prompts can cause instability or texture collapse.
- May struggle to fully suppress certain concepts, especially stylistic cues.
- Hyperparameters (phi, tau, alpha) are tuned per model family.
Experimental setup
Datasets: COCO-5K Pick-a-Pic v2
Benchmarks: COCO-5K
Metrics: CLIP Score FID PFID ImageReward user preference
Baselines: CFG NASA PAG CFG+PAG
Builds on: Classifier-Free Guidance NASA Perturbed Attention Guidance Flow Matching
Closest papers by meaning
- Dynamic Negative Guidance of Diffusion Models (ICLR 2025 · similarity 0.91)
- Feedback Guidance of Diffusion Models (NeurIPS 2025 · similarity 0.90)
- Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Models (AAAI 2025 · similarity 0.90)
- REG: Rectified Gradient Guidance for Conditional Diffusion Models (ICML 2025 · similarity 0.89)
- Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models (NeurIPS 2024 · similarity 0.89)
- Diffusion-NPO: Negative Preference Optimization for Better Preference Aligned Generation of Diffusion Models (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.