Research map / Image generation / Fast diffusion sampling
RAD: Region-Aware Diffusion Models for Image Inpainting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RAD, a diffusion model reformulation with pixel-wise noise schedules (Perlin-noise masks, spatial noise embedding, LoRA fine-tuning of ADM) that enables fast, training-based image inpainting without extra modules.
Problem
Diffusion-based image inpainting either hijacks the reverse process of pretrained models with nested loops (slow) or adds conditioning modules, both adding complexity and inference cost.
Approach
Assign each pixel its own noise schedule so that masked regions are noised and denoised while unmasked regions are already clean, letting the plain reverse process inpaint asynchronously; the pixel-wise noise level is fed to the network via a spatial noise embedding replacing the timestep embedding, and training is done cheaply with LoRA on pretrained ADMs.
Key results
- RAD achieves FID 22.1 (box) and LPIPS 0.074 on FFHQ box masks, outperforming RePaint (25.7 FID) and MCG (23.7 FID) as reported.
- RAD inference takes 8.4 s per 256x256 image on a TITAN RTX versus 837 s for RePaint and 128 s for MCG, about 100x and 15x faster.
- Ablation on FFHQ shows that applying RAD reverse steps to a pretrained ADM without training degrades FID to 128.3 (box), confirming the value of spatially variant training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add spatially variant noise training and spatial noise embedding instead of using RAD reverse steps on an unmodified pretrained ADM (ablation) | RAD on FFHQ · FFHQ Box | FID: 128.3 -> 22.1 | Helped |
| Add spatially variant noise training and spatial noise embedding instead of using RAD reverse steps on an unmodified pretrained ADM (ablation) | RAD on FFHQ · FFHQ Extreme | FID: 172.0 -> 33.4 | Helped |
| Add spatially variant noise training and spatial noise embedding instead of using RAD reverse steps on an unmodified pretrained ADM (ablation) | RAD on FFHQ · FFHQ Wide | FID: 95.7 -> 21.5 | Helped |
| Add spatially variant noise training and spatial noise embedding instead of using RAD reverse steps on an unmodified pretrained ADM (ablation) | RAD on FFHQ · FFHQ Box | LPIPS: 0.279 -> 0.074 | Helped |
| Add spatially variant noise training and spatial noise embedding instead of using RAD reverse steps on an unmodified pretrained ADM (ablation) | RAD on FFHQ · FFHQ Extreme | LPIPS: 0.441 -> 0.317 | Helped |
| Add spatially variant noise training and spatial noise embedding instead of using RAD reverse steps on an unmodified pretrained ADM (ablation) | RAD on FFHQ · FFHQ Wide | LPIPS: 0.240 -> 0.078 | Helped |
| Add spatial noise embedding instead of conventional t embedding (ablation) | RAD on FFHQ · FFHQ Box | FID: 23.5 -> 22.1 | Helped |
| Add spatial noise embedding instead of conventional t embedding (ablation) | RAD on FFHQ · FFHQ Extreme | FID: 37.0 -> 33.4 | Helped |
| Add spatial noise embedding instead of conventional t embedding (ablation) | RAD on FFHQ · FFHQ Wide | FID: 26.8 -> 21.5 | Helped |
| Add spatial noise embedding instead of conventional t embedding (ablation) | RAD on FFHQ · FFHQ Box | LPIPS: 0.086 -> 0.074 | Helped |
| Add spatial noise embedding instead of conventional t embedding (ablation) | RAD on FFHQ · FFHQ Extreme | LPIPS: 0.333 -> 0.317 | Helped |
| Add spatial noise embedding instead of conventional t embedding (ablation) | RAD on FFHQ · FFHQ Wide | LPIPS: 0.085 -> 0.078 | Helped |
Surprising findings
- Applying RAD's reverse steps to a pretrained ADM without retraining performs very poorly, indicating that training with spatially variant noise is essential.
Limitations
- Requires fresh training of the altered formulation (mitigated with LoRA), so it depends on mask distribution.
- RePaint or MCG may give better results in drastic cases despite much longer inference time.
- Evaluated only on FFHQ, LSUN Bedroom, and ImageNet at 256x256 with FID and LPIPS.
Experimental setup
Datasets: FFHQ LSUN Bedroom ImageNet
Benchmarks: FFHQ LSUN Bedroom ImageNet
Metrics: FID LPIPS inference time
Baselines: LaMa Score-SDE DDRM RePaint MCG DDNM DeqIR
Builds on: DDPM ADM LoRA RePaint DDIM
Closest papers by meaning
- Image Inpainting via Tractable Steering of Diffusion Models (ICLR 2024 · similarity 0.90)
- TD-Paint: Faster Diffusion Inpainting Through Time-Aware Pixel Conditioning (ICLR 2025 · similarity 0.89)
- Structure Matters: Tackling the Semantic Discrepancy in Diffusion Models for Image Inpainting (CVPR 2024 · similarity 0.89)
- InpDiffusion: Image Inpainting Localization via Conditional Diffusion Models (AAAI 2025 · similarity 0.88)
- RepLDM: Reprogramming Pretrained Latent Diffusion Models for High-Quality, High-Efficiency, High-Resolution Image Generation (NeurIPS 2025 · similarity 0.88)
- DesignEdit: Unify Spatial-Aware Image Editing via Training-free Inpainting with a Multi-Layered Latent Diffusion Framework (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.