Research map / Image generation / Fast diffusion sampling
FreeU: Free Lunch in Diffusion U-Net
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FreeU, a training-free inference-time method that re-weights diffusion U-Net backbone and skip-connection features (scaling factors b and s) to improve sample quality of Stable Diffusion, SDXL, ModelScope and other diffusion models.
Problem
Diffusion U-Net skip connections carry high-frequency information that can weaken the backbone's denoising ability at inference, degrading generation quality; the internal roles of the U-Net components are under-explored.
Approach
Analyzes U-Net backbone vs. skip features in the Fourier domain, finding the backbone drives denoising and skips inject high-frequency detail; FreeU amplifies backbone features (structure-aware, half-channel scaling) and attenuates low-frequency components of skip features at inference, with no training.
Key results
- In a 120-participant study, SD+FreeU received 84.58% of votes for image-text alignment and 86.27% for image quality versus SD's 15.42% and 13.73%.
- For text-to-video, ModelScope+FreeU received 84.68% of votes for video-text alignment and 85.75% for video quality.
- Fourier analysis shows FreeU reduces high-frequency content of denoised outputs at each denoising step.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add FreeU to Stable Diffusion | Stable Diffusion · Text-to-image human evaluation | Image-text alignment votes: 15.42% -> 84.58% | Helped |
| Add FreeU to Stable Diffusion | Stable Diffusion · Text-to-image human evaluation | Image quality votes: 13.73% -> 86.27% | Helped |
| Add FreeU to ModelScope | ModelScope · Text-to-video human evaluation | Video-text alignment votes: 15.32% -> 84.68% | Helped |
| Add FreeU to ModelScope | ModelScope · Text-to-video human evaluation | Video quality votes: 14.25% -> 85.75% | Helped |
| Increase the backbone feature scaling factor b (ablation) | Diffusion U-Net · Image generation | Generation quality | Helped |
| Increase the skip feature scaling factor s directly (ablation) | Diffusion U-Net · Image generation | Generation quality | No clear effect |
| Scale all backbone feature channels instead of half (ablation) | FreeU · Image generation | Texture quality | Hurt |
| Add half-channel backbone scaling (ablation) | FreeU · Image generation | Generation quality | Helped |
| Add skip feature scaling to backbone scaling (ablation) | SD+FreeU (b) · Image generation | Image realism | Helped |
| Use a structure-related adaptive backbone factor instead of a constant factor (ablation) | FreeU · Image generation | Texture quality | Helped |
Surprising findings
- Scaling skip-connection features has only limited influence on generation quality, while scaling backbone features strongly improves it.
- Scaling all backbone channels causes oversmoothed textures, so scaling only half the channels is used.
Limitations
- Evaluation relies mainly on a small user study rather than standard automatic metrics.
- Scaling factors b and s are set by hand and must be tuned per model.
Experimental setup
Metrics: user preference vote percentage (image-text alignment, image
Baselines: Stable Diffusion 1.4 SDXL ModelScope
Builds on: Stable Diffusion U-Net DDPM
Closest papers by meaning
- U-Nets as Belief Propagation: Efficient Classification, Denoising, and Diffusion in Generative Hierarchical Models (ICLR 2025 · similarity 0.88)
- Not All Parameters Matter: Masking Diffusion Models for Enhancing Generation Ability (CVPR 2025 · similarity 0.88)
- UDT: Reconciling U-Nets and Diffusion Transformers with Data-Adaptive Token Reduction (NeurIPS 2026 · similarity 0.88)
- ALTER: All-in-One Layer Pruning and Temporal Expert Routing for Efficient Diffusion Generation (NeurIPS 2025 · similarity 0.87)
- U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformers (NeurIPS 2024 · similarity 0.87)
- UniCon: Unidirectional Information Flow for Effective Control of Large-Scale Diffusion Models (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.