Research map / Image generation / Fast diffusion sampling
IMPUS: Image Morphing with Perceptually-Uniform Sampling Using Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IMPUS, a diffusion-based image morphing method that interpolates optimized text embeddings and DDIM latent states, adapts the model with LoRA whose rank is chosen by a relative perceptual path diversity score, and uses LPIPS-uniform sampling for smooth morphs.
Problem
Generating smooth, direct, and realistic interpolation sequences between two arbitrary natural images, where endpoint embeddings may lie on distinct conditional distributions of a latent diffusion model.
Approach
Interpolate linearly in optimized CLIP text-embedding space and slerp in the DDIM/probability-flow latent space, adapt the diffusion model to the endpoint pair with LoRA whose rank is set by a heuristic bottleneck based on a relative perceptual path diversity (rPPD) score, and sample alpha values so consecutive outputs have constant LPIPS difference.
Key results
- On the comparison setting of prior work, IMPUS reduces PPL_1/16 from 143.25 to 92.58 and FID from 198.16 to 148.45.
- Model adaptation with LoRA lowers FID on CelebA-HQ from 52.76 to 43.59 and total LPIPS from 1.45 to 1.07 versus no adaptation.
- Heuristic rank selection via rPPD yields competitive directness and FID compared with fixed ranks 4 and 32 on benchmark subsets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add model adaptation (ablation) | IMPUS · CelebA-HQ | LPIPST: 1.45 ± 0.05 -> 1.07 ± 0.07 | Helped |
| add model adaptation (ablation) | IMPUS · AFHQ | FID: 42.67 ± 12.39 -> 36.79 ± 11.87 | Helped |
| add model adaptation (ablation) | IMPUS · LSUN | LPIPST: 1.94 -> 1.79 | Helped |
| replace textual inversion with a category prompt (ablation) | IMPUS · female↔male | endpoint reconstruction error: 0.081 (0.137) -> 0.092 (0.308) | Hurt |
| replace textual inversion with a category prompt (ablation) | IMPUS · cat↔dog | endpoint reconstruction error: 0.119 (0.167) -> 0.125 (0.206) | Hurt |
| replace textual inversion with a category prompt (ablation) | IMPUS · church↔church | endpoint reconstruction error: 0.093 (0.248) -> 0.113 (0.412) | Hurt |
| use separate LoRA for unconditional score estimates instead of randomly discarding conditioning (ablation) | IMPUS · CelebA female-male pairs | maximum endpoint LPIPS reconstruction error: 0.211 -> 0.137 | Helped |
| use perceptually-uniform sampling instead of uniform sampling (ablation) | IMPUS · image morphing sequences | transition smoothness | Helped |
| use LoRA adaptation instead of vanilla model adaptation (ablation) | IMPUS · image morphing sequences | image fidelity | Helped |
| use heuristic LoRA rank instead of fixed rank 4 (ablation) | IMPUS · image morphing sequences | directness | Helped |
| use spherical interpolation instead of linear interpolation for latent states (ablation) | IMPUS · image morphing sequences | intermediate sample naturalness | Helped |
Surprising findings
- Higher LoRA ranks performed better on benchmark datasets, contrary to the trend observed on internet-collected images.
- Endpoint reconstruction error did not decrease as LoRA rank increased, attributed to numerical error in ODE inversion.
Limitations
- Abrupt transitions occur when endpoint images are semantically very different.
- Generates incorrect hands in some cases.
- FID is an imperfect metric for morph sequences because samples are limited and non-i.i.d.
Experimental setup
Datasets: CelebA-HQ AFHQ LSUN church internet images 25 image pairs from prior work
Benchmarks: CelebA-HQ AFHQ LSUN church
Metrics: total LPIPS max LPIPS FID PPL endpoint reconstruction LPIPS
Baselines: Wang et al. diffusion-based morphing (ControlNet-based) DiffMorph Automatic Image Morphing Neural Crossbreed
Builds on: Stable Diffusion textual inversion DDIM LoRA LPIPS
Closest papers by meaning
- AID: Attention Interpolation of Text-to-Image Diffusion (NeurIPS 2024 · similarity 0.88)
- Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models (CVPR 2024 · similarity 0.87)
- Boosting Generative Image Modeling via Joint Image-Feature Synthesis (NeurIPS 2025 · similarity 0.87)
- Würstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models (ICLR 2024 · similarity 0.87)
- PAIR Diffusion: A Comprehensive Multimodal Object-Level Image Editor (CVPR 2024 · similarity 0.87)
- DiffMorpher: Unleashing the Capability of Diffusion Models for Image Morphing (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.