Research map / Image generation / Fast diffusion sampling
Learning normalized image densities via dual score matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns a normalized energy model of images by combining denoising (space) score matching with a novel time-score matching objective over noise levels, using an energy network built as an inner product of a homogeneous score network's input and output.
Problem
Learning normalized probability (energy) models of high-dimensional images is hard due to the intractable normalization constant and the curse of dimensionality; single score matching recovers energies only up to mode-dependent constants.
Approach
Define an energy network U(y,t) = 1/2 <y, s(y,t)> from a homogeneous score network so its spatial gradient equals the score, and train jointly with a denoising objective (space score) and a secondary time-score matching objective that enforces consistency across noise levels and fixes the global normalization constant.
Key results
- Achieves 3.36 bits/dimension NLL on ImageNet64 test set, comparable to state-of-the-art methods such as TarFlow (2.99) and NFDM (3.20).
- Energy network denoising PSNR matches or slightly exceeds the score network across noise levels (e.g., 35.67 vs 35.58 dB at variance 30).
- Two models trained on disjoint halves converge to nearly identical log probabilities at N=1e5 training images.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the energy network trained with dual score matching instead of the score network trained with standard denoising score matching | UNet on ImageNet64 · ImageNet64, noise variance 90 and 75 versus 60 through -30 | PSNR (dB): 90.20 vs 90.09; 75.47 vs 75.17; 60.43 vs 60.45; 47.19 vs 47.25; 35.58 vs 35.67; 25.92 vs 26.01; 18.84 vs 18.88; 13.31 vs | Mixed |
| Train with dual score matching rather than single (space) score matching (ablation) | Energy model on the 1000-dimensional Gaussian scale mixture · Gaussian scale mixture, energy recovery across modes | energy accuracy | Helped |
| Tune a tradeoff hyperparameter between the space-score and time-score objectives instead of adding the normalized objectives directly (ablation) | Dual score matching · ImageNet64 | NLL | No clear effect |
Surprising findings
- Log probabilities of ImageNet images follow a Gumbel distribution, and the energy spans a range of 34.4 dB/dim across the test set.
- Effective dimensionality of local neighborhoods varies greatly across images, with some high-probability images having low-dimensional neighborhoods even at small scales.
Limitations
- Training is roughly 4x slower than the base score network due to double backpropagation.
- No theoretical guarantee that the objective yields an accurate normalized energy; the model may not be exactly normalized and out-of-distribution behavior may be unreliable.
- NLL comparisons with other methods are approximate because of differing preprocessing, dequantization, and estimator types.
Experimental setup
Datasets: ImageNet64 CelebA CIFAR-10
Benchmarks: ImageNet64
Metrics: negative log likelihood (bits/dimension) PSNR denoising energy generalization error
Baselines: Glow PixelCNN I-DDPM VDM FM NFDM TarFlow single score matching
Builds on: denoising score matching diffusion models Miyasawa-Tweedie identity
Closest papers by meaning
- Learning Energy-Based Models by Cooperative Diffusion Recovery Likelihood (ICLR 2024 · similarity 0.87)
- Learning Energy-Based Models from Stochastic Interpolants using Spatiotemporal Differences (NeurIPS 2026 · similarity 0.87)
- Generalization in diffusion models arises from geometry-adaptive harmonic representations (ICLR 2024 · similarity 0.87)
- Dual-Rate Diffusion: Accelerating diffusion models with an interleaved heavy-light network (NeurIPS 2026 · similarity 0.87)
- Information Theoretic Learning for Diffusion Models with Warm Start (NeurIPS 2025 · similarity 0.86)
- Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalization (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.