Research map / Probabilistic and scientific modeling / Variational and generative sampling
Likelihood Training of Cascaded Diffusion Models via Hierarchical Volume-preserving Maps
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes probabilistic cascaded diffusion models (PCDM) on latent spaces from hierarchical volume-preserving maps (Laplacian pyramid and Haar wavelet), giving exact joint likelihoods over scales and state-of-the-art bits-per-dimension, OOD detection, and lossless compression results.
Problem
Cascaded (multi-scale) diffusion models produce high-quality samples but their likelihood over the finest scale is intractable, since exact evaluation requires marginalizing intermediate scales.
Approach
Define hierarchical volume-preserving maps (Jacobian with unit volume factor) so that the likelihood of the data equals the joint likelihood of the scales; instantiate with Laplacian pyramid (tight frame) and Haar wavelet (orthogonal), then train a cascaded VDM-style diffusion model on these scales with a variational bound. Theoretically links the bound to an EMD-based score matching objective com
Key results
- W-PCDM achieves 2.35 BPD on CIFAR10 and 2.64 BPD on ImageNet 128x128, state-of-the-art among compared likelihood models.
- W-PCDM reaches average OOD AUROC of 0.92 versus 0.83 for VDM on CIFAR10-trained OOD detection.
- Lossless compression on CIFAR10 reaches 2.37 BPD with W-PCDM, below ARDM (2.71) and VDM (2.72).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a wavelet hierarchy instead of a Laplacian pyramid hierarchy | PCDM · CIFAR10 32x32 test | BPD: 2.36 -> 2.35 | Helped |
| use a wavelet hierarchy instead of a Laplacian pyramid hierarchy | PCDM · ImageNet 32x32 test | BPD: 3.52 -> 3.32 | Helped |
| use a wavelet hierarchy instead of a Laplacian pyramid hierarchy | PCDM · ImageNet 64x64 test | BPD: 3.12 -> 2.95 | Helped |
| use a wavelet hierarchy instead of a Laplacian pyramid hierarchy | PCDM · ImageNet 128x128 test | BPD: 2.91 -> 2.64 | Helped |
| use a wavelet hierarchy instead of a Laplacian pyramid hierarchy | PCDM · CIFAR10 lossless compression | Compression BPD: 2.40 -> 2.37 | Helped |
| replace the volume-preserving map with a standard non-volume-preserving hierarchy (ablation) | PCDM · CIFAR10 32x32 test | BPD: 2.35 -> 3.20 | Hurt |
| replace the volume-preserving map with a standard non-volume-preserving hierarchy (ablation) | PCDM · ImageNet 32x32 test | BPD: 3.32 -> 4.21 | Hurt |
| replace the volume-preserving map with a standard non-volume-preserving hierarchy (ablation) | PCDM · ImageNet 64x64 test | BPD: 2.95 -> 3.87 | Hurt |
| replace the volume-preserving map with a standard non-volume-preserving hierarchy (ablation) | PCDM · ImageNet 128x128 test | BPD: 2.64 -> 3.75 | Hurt |
| replace the volume-preserving map with a standard non-volume-preserving hierarchy (ablation) | PCDM · CIFAR10 to SVHN | AUROC: 0.74 -> 0.71 | No clear effect |
| replace the volume-preserving map with a standard non-volume-preserving hierarchy (ablation) | PCDM · CIFAR10 OOD average | AUROC: 0.92 -> 0.90 | Hurt |
| replace the volume-preserving map with a standard non-volume-preserving hierarchy (ablation) | PCDM · CIFAR10 lossless compression | Compression BPD: 2.37 -> 3.22 | Hurt |
Surprising findings
- Removing the volume-preserving property (C-VDM) substantially degrades BPD (e.g., 3.20 vs 2.35 on CIFAR10), showing the non-volume-preserving hierarchy is much worse despite similar architecture.
Limitations
- Bits-back compression requires significant initialization overhead and fixed encoding/decoding order, making single-image compression expensive.
- Theoretical link to EMD score matching relies on a constant alpha and beta whose values are unspecified; open question whether EMD score matching keeps statistical guarantees.
- Non-volume-preserving variant performs much worse, so gains depend on restricting to volume-preserving maps.
Experimental setup
Datasets: CIFAR10 ImageNet 32x32 ImageNet 64x64 ImageNet 128x128 SVHN
Benchmarks: CIFAR10 ImageNet 32x32 ImageNet 64x64 ImageNet 128x128 SVHN OOD CIFAR10 lossless compression
Metrics: bits per dimension FID AUROC
Baselines: VDM C-VDM Flow++ Sparse Transformer Very Deep VAE Flow Matching Score Flow i-DODE PixelCNN++ Glow ARDM IDF
Builds on: VDM Variational diffusion models Improved DDPM Cascaded diffusion models Bits-Back ANS
Closest papers by meaning
- Diffusion Priors for Variational Likelihood Estimation and Image Denoising (NeurIPS 2024 · similarity 0.87)
- Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood (ICLR 2024 · similarity 0.87)
- DiffEnc: Variational Diffusion with a Learned Encoder (ICLR 2024 · similarity 0.87)
- Information Theoretic Learning for Diffusion Models with Warm Start (NeurIPS 2025 · similarity 0.87)
- Nested Diffusion Models Using Hierarchical Latent Priors (CVPR 2025 · similarity 0.87)
- Diffusion Models With Learned Adaptive Noise (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.