Research map / Image generation / Editing and style transfer
DiffMorpher: Unleashing the Capability of Diffusion Models for Image Morphing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
DiffMorpher morphs between two real or generated images by fitting a LoRA to each image and interpolating LoRA weights, latent noises (slerp), attention features, and text conditions on Stable Diffusion v2.1-base, plus AdaIN adjustment and a reschedule sampling schedule.
Problem
Diffusion models produce abrupt, semantically inconsistent content changes when naively interpolated between two images, unlike GANs whose latent spaces interpolate smoothly, making high-quality image morphing hard.
Approach
Fit one LoRA per input image so the semantics of each image live in a low-rank parameter space that can be linearly interpolated, combine this with slerp of DDIM-inverted latent noises, self-attention K/V interpolation and injection, AdaIN latent statistic adjustment, and a histogram-equalization-based reschedule of interpolation ratios, all without training a new model.
Key results
- On MorphBench, DiffMorpher reaches overall FID 54.69 and PPL 21.10 with PDV 21.42, versus DDIM at 101.68/25.38/288.51 and Diff.Interp. at 105.23/112.84 for overall FID/PPL/PDV.
- Warp & Blend obtains lower PPL (14.27) and PDV (3.67) than DiffMorpher but is noted to produce ghosting artifacts.
- Ablation shows LoRA interpolation cuts DDIM's FID from 101.68 to 44.40, and the reschedule plus AdaIN step lowers PDV from 157.73 to 21.42 in the full model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DiffMorpher instead of Warp & Blend | MorphBench image morphing · MorphBench Overall | FID: 67.57 -> 54.69 | Helped |
| use DiffMorpher instead of Warp & Blend | MorphBench image morphing · MorphBench Overall | PPL: 14.27 -> 21.10 | Hurt |
| use DiffMorpher instead of Warp & Blend | MorphBench image morphing · MorphBench Overall | PDV: 3.67 -> 21.42 | Hurt |
| add LoRA interpolation to DDIM (ablation) | DDIM image morphing · MorphBench Overall | FID: 101.68 -> 44.40 | Helped |
| add LoRA interpolation to DDIM (ablation) | DDIM image morphing · MorphBench Overall | PPL: 25.38 -> 21.81 | Helped |
| add LoRA interpolation to DDIM (ablation) | DDIM image morphing · MorphBench Overall | PDV: 288.51 -> 249.33 | Helped |
| add attention interpolation and replacement to LoRA interpolation (ablation) | DiffMorpher · MorphBench Overall | PPL: 21.81 -> 19.86 | Helped |
| add attention interpolation and replacement to LoRA interpolation (ablation) | DiffMorpher · MorphBench Overall | PDV: 249.33 -> 157.73 | Helped |
| add AdaIN adjustment and rescheduling to LoRA and attention interpolation (ablation) | DiffMorpher · MorphBench Overall | PDV: 157.73 -> 21.42 | Helped |
| increase attention replacement strength from λ=0 to λ=1 (ablation) | DiffMorpher · MorphBench Overall | PPL: 23.85 -> 17.85 | Helped |
| increase attention replacement strength from λ=0 to λ=1 (ablation) | DiffMorpher · MorphBench Overall | PDV: 81.15 -> 12.86 | Helped |
| increase attention replacement strength from λ=0 to λ=1 (ablation) | DiffMorpher · MorphBench Overall | FID: 53.78; 55.84 | Mixed |
Surprising findings
- Warp & Blend achieves better PPL and PDV than DiffMorpher, which the authors attribute to smooth linear warping that still shows ghosting artifacts.
- Rescheduling raises FID slightly (44.90 to 54.69), since non-rescheduled results are biased toward the input images.
Limitations
- Attention replacement in late denoising steps causes blur, so the lambda hyperparameter must be tuned (0.4-0.6 works in most cases)
- Warp & Blend has better PPL and PDV than the proposed method
- Each image requires fitting its own LoRA before morphing
Experimental setup
Datasets: MorphBench (constructed for evaluation)
Benchmarks: MorphBench
Metrics: FID PPL PDV
Baselines: Warp & Blend DGP StyleGAN-XL DDIM Diff.Interp.
Builds on: LoRA DDIM inversion Stable Diffusion Diff.Interp.
Closest papers by meaning
- Diffusing Differentiable Representations (NeurIPS 2024 · similarity 0.87)
- IMPUS: Image Morphing with Perceptually-Uniform Sampling Using Diffusion Models (ICLR 2024 · similarity 0.87)
- Structure Matters: Tackling the Semantic Discrepancy in Diffusion Models for Image Inpainting (CVPR 2024 · similarity 0.86)
- DCTdiff: Intriguing Properties of Image Generative Modeling in the DCT Space (ICML 2025 · similarity 0.85)
- Diffusion Models Without Attention (CVPR 2024 · similarity 0.85)
- RAD: Region-Aware Diffusion Models for Image Inpainting (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.