Research map / Image generation / Image restoration
NoiseDiffusion: Correcting Noise for Image Interpolation with Diffusion Models beyond Spherical Linear Interpolation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NoiseDiffusion, a noise-correction method for interpolating natural images with diffusion models that combines clipped latent slerp-style mixing, injected Gaussian noise, and raw-image injection in noisy space.
Problem
Spherical linear interpolation (slerp) of diffusion latents yields artifacts when interpolating natural (non-generated) images because the encoded noise is not valid Gaussian noise, while introducing noise for interpolation loses original image information.
Approach
Corrects the encoded noise by clipping extreme latent values, adds subtle Gaussian noise under a norm constraint (alpha^2+beta^2+gamma^2=1) justified by high-dimensional hypersphere concentration, and compensates lost information by injecting raw image content (mu, nu coefficients) into the noisy-space interpolation.
Key results
- Qualitative comparison on natural images shows slerp introduces visible artifacts and noise-introduction methods add extra information, while NoiseDiffusion reduces artifacts and preserves original features (no quantitative metrics reported).
- Lowering the clipping boundary reduces artifacts substantially but causes feature loss and darkening; applying clipping before and after interpolation works best among the three variants.
- Adjusting mu and nu compensates for the information loss and darkening caused by boundary control.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NoiseDiffusion instead of spherical linear interpolation | Natural-image interpolation with diffusion models · LSUN Cat-256 and LSUN Bedroom-256 | image quality and feature preservation | Helped |
| Use NoiseDiffusion instead of spherical linear interpolation | Natural-image interpolation with Stable Diffusion · Stable Diffusion | image quality and feature preservation | Helped |
| Use NoiseDiffusion instead of directly introducing Gaussian noise | Natural-image interpolation with diffusion models · LSUN Cat-256 and LSUN Bedroom-256 | original image feature preservation | Helped |
| Use NoiseDiffusion instead of directly introducing Gaussian noise | Natural-image interpolation with Stable Diffusion · Stable Diffusion | image information preservation | Helped |
| Increase the Gaussian-noise lubrication coefficient γ (ablation) | NoiseDiffusion · Natural-image interpolation | image artifacts | Helped |
| Increase the Gaussian-noise lubrication coefficient γ (ablation) | NoiseDiffusion · Natural-image interpolation | original image feature preservation | Hurt |
| Decrease the clipping boundary (ablation) | NoiseDiffusion · Natural-image interpolation | image artifacts | Helped |
| Decrease the clipping boundary (ablation) | NoiseDiffusion · Natural-image interpolation | image information preservation | Hurt |
| Apply clipping before and after interpolation rather than only before or only after (ablation) | NoiseDiffusion · Natural-image interpolation | image artifacts | Helped |
| Vary the original-image information coefficients μ and ν (ablation) | NoiseDiffusion · Natural-image interpolation | image brightness | Mixed |
Surprising findings
- Naively standardizing latent mean and variance via translation and scaling fails to improve quality and loses image information.
Limitations
- Encoding step roughly doubles processing time compared with directly introducing noise.
- Effectiveness validated only on image data, not other modalities.
- Evaluation is qualitative with no quantitative metrics reported.
Experimental setup
Datasets: LSUN Cat-256 LSUN Bedroom-256
Baselines: spherical linear interpolation introducing noise for interpolation (SDEdit-based)
Builds on: SDEdit score-based SDE probability flow ODE spherical linear interpolation
Closest papers by meaning
- Training-Free Adaptive Diffusion with Bounded Difference Approximation Strategy (NeurIPS 2024 · similarity 0.89)
- How I Warped Your Noise: a Temporally-Correlated Noise Prior for Diffusion Models (ICLR 2024 · similarity 0.88)
- Denoising Diffusion Bridge Models (ICLR 2024 · similarity 0.88)
- Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignment (NeurIPS 2024 · similarity 0.88)
- Diffusion Priors for Variational Likelihood Estimation and Image Denoising (NeurIPS 2024 · similarity 0.87)
- An Edit Friendly DDPM Noise Space: Inversion and Manipulations (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.