Research map / Image generation / Autoregressive and latent generation
Isometric Representation Learning for Disentangled Latent Space of Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Isometric Diffusion, which fine-tunes diffusion models with an isometry regularizer on the encoder's Jacobian (stereographic coordinates on a hypersphere) to obtain a more disentangled latent space for interpolation, inversion, and linear editing.
Problem
Latent spaces of diffusion models are entangled and distorted, so linear or spherical interpolation, DDIM inversion, and attribute editing produce abrupt, multi-factor changes.
Approach
Treat the noise latent as a hypersphere with stereographic coordinates and add a scale-free isometry loss (trace-based, estimated stochastically) on the score model's encoder at each timestep after a ratio gamma, rather than an EMA-based path length regularizer or full pullback-metric computation.
Key results
- On CelebA-HQ with DDPM, the isometry loss reduces PPL-50k from 648 to 455 while FID-10k rises only slightly from 15.89 to 16.18.
- Versus path length regularizer, the isometric loss with the stereographic metric gives FID 16.18 and PPL 455, while the path length regularizer gives FID 20.04 and PPL 552.
- On CelebA-HQ LDM, LS drops from 2.38 to 2.15 and VoR from 1.04 to 0.54 with the isometric loss.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add isometric loss | DDPM · CIFAR-10 | PPL-50k: 126 -> 101 | Helped |
| Add isometric loss | DDPM · CIFAR-10 | FID-10k: 10.19 -> 10.50 | Mixed |
| Add isometric loss | DDPM · LSUN-Church | PPL-50k: 2028 -> 1559 | Helped |
| Add isometric loss | DDPM · LSUN-Church | FID-10k: 10.56 -> 12.10 | Hurt |
| Add isometric loss | DDPM · LSUN-Bedrooms | PPL-50k: 4515 -> 3809 | Helped |
| Add isometric loss | DDPM · CelebA-HQ | LS: 1.91 -> 1.51 | Helped |
| Use path length regularizer instead of isometric loss (ablation) | DDPM · CelebA-HQ | PPL-50k: 648 -> 552 | Helped |
| Use path length regularizer instead of isometric loss (ablation) | DDPM · CelebA-HQ | FID-10k: 15.89 -> 20.04 | Hurt |
| Use isometric loss with Euclidean metric G = I (ablation) | DDPM · CelebA-HQ | PPL-50k: 648 -> 619 | Helped |
| Use isometric loss with Euclidean metric G = I (ablation) | DDPM · CelebA-HQ | FID-10k: 15.89 -> 16.60 | Hurt |
| Use the spherical Riemannian metric G_s instead of G = I (ablation) | DDPM · CelebA-HQ | PPL-50k: 619 -> 455 | Helped |
| Increase lambda_iso from 10^{-4} to 10^{-3} (ablation) | DDPM with gamma 0.5 and G = I · CelebA-HQ | FID-10k: 16.60 -> 30.28 | Hurt |
Surprising findings
- Applying the isometry loss with the Euclidean metric (G=I) raises FID to 30.28 at gamma=0.5 with lambda 1e-3, showing the choice of metric matters.
- On CIFAR-10 and LSUN-Church/Bedrooms, the isometric loss slightly worsens FID (e.g., LSUN-Church 10.56 to 12.10).
Limitations
- Trades off FID against disentanglement metrics; FID worsens on several datasets.
- Experiments are on unconditional models and small-resolution datasets; conditional/text-to-image models are left to future work.
- Stochastic trace estimator introduces approximation error.
Experimental setup
Datasets: CIFAR-10 CelebA-HQ LSUN-Church LSUN-Bedrooms
Benchmarks: CIFAR-10 CelebA-HQ LSUN-Church LSUN-Bedrooms
Metrics: FID-10k PPL-50k mRTL MCN VoR LS MSE PSNR SSIM LPIPS
Baselines: DDPM LDM path length regularizer (StyleGAN2) ADM
Builds on: path length regularizer DDPM DDIM h-space (diffusion autoencoders semantic space) isometric regularization for autoencoders
Closest papers by meaning
- What Matters for Diffusion-Friendly Latent Manifold? Prior-Aligned Autoencoders for Latent Diffusion (NeurIPS 2026 · similarity 0.91)
- Can Diffusion Models Disentangle? A Theoretical Perspective (NeurIPS 2025 · similarity 0.90)
- DisCo-Diff: Enhancing Continuous Diffusion Models with Discrete Latents (ICML 2024 · similarity 0.89)
- How Diffusion Models Learn to Factorize and Compose (NeurIPS 2024 · similarity 0.89)
- Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models (CVPR 2024 · similarity 0.88)
- Diffusion Model with Cross Attention as an Inductive Bias for Disentanglement (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.