Research map / Probabilistic and scientific modeling / Variational and generative sampling
DiffEnc: Variational Diffusion with a Learned Encoder
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffEnc, a variational diffusion model with a time-dependent learned encoder that shifts the mean of the forward process (used only in training) plus a free generative-variance weight, improving CIFAR-10 likelihood over VDM.
Problem
Standard diffusion models use a fixed, simple forward process and equal forward/generative noise variances; this limits the flexibility of the variational bound and the achievable likelihood.
Approach
Replace the fixed forward-process mean with a learned, data- and log-SNR-dependent encoder x_phi(lambda_t) used only during training, add a counterterm in the generative mean to cancel the extra gradient term in the continuous-time loss, and analyze relaxing the equal-variance assumption via a weight w_t, showing ELBO diverges in continuous time unless the weight is 1.
Key results
- DiffEnc-32-4 achieves 2.620 BPD on CIFAR-10 versus 2.641 for VDMv-32 (3 seeds), with a reported p-value of 0.03 for the improvement.
- On ImageNet32, DiffEnc Trainable and VDM with v-parameterization both reach 3.46 BPD, so no improvement is seen there.
- With a fixed noise schedule, the trainable encoder reduces latent loss but does not change total loss for small models; DiffEnc-8-nt is consistently worse on diffusion loss than the baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a trainable time-dependent encoder | VDMv-32 · CIFAR-10 | total BPD: 2.641 ± 0.003 -> 2.620 ± 0.006 | Helped |
| Add a trainable time-dependent encoder | VDMv-32 · CIFAR-10 | diffusion loss: 2.629 ± 0.003 -> 2.609 ± 0.006 | Helped |
| Add a trainable time-dependent encoder | VDMv-32 · CIFAR-10 | latent loss: 0.0012 ± 0.0 -> 0.0007 ± (3 × 10^{-6}) | Helped |
| Add a trainable time-dependent encoder | VDMv · ImageNet32 | BPD: 3.46 -> 3.46 | No clear effect |
| Use v-parameterization instead of epsilon-parameterization | VDM · CIFAR-10 | likelihood BPD: 2.65 -> 2.64 | Helped |
| Replace VDMv-8 with a small trainable encoder (ablation) | DiffEnc-8-2 · CIFAR-10, fixed noise schedule | total BPD: 2.783\pm 0.004 -> 2.783± 0.004 | No clear effect |
| Replace VDMv-8 with a small trainable encoder (ablation) | DiffEnc-8-2 · CIFAR-10, fixed noise schedule | latent loss: 0.0012\pm 0.0 -> 0.0006\pm (3 \times 10^{-5}) | Helped |
| Replace VDMv-8 with a small trainable encoder (ablation) | DiffEnc-8-2 · CIFAR-10, fixed noise schedule | diffusion loss: 2.772\pm 0.004 -> 2.772± 0.004 | No clear effect |
| Replace the trainable encoder with a non-trainable encoder (ablation) | DiffEnc-8-nt · CIFAR-10, fixed noise schedule | total BPD: 2.783\pm 0.004 -> 2.789\pm 0.004 | Hurt |
| Replace the trainable encoder with a non-trainable encoder (ablation) | DiffEnc-8-nt · CIFAR-10, fixed noise schedule | diffusion loss: 2.772\pm 0.004 -> 2.779\pm 0.004 | Hurt |
| Replace a fixed noise schedule with trainable noise endpoints (ablation) | DiffEnc-8-2 · CIFAR-10 | total BPD: 2.783\pm 0.004 -> 2.783\pm 0.003 | No clear effect |
Surprising findings
- The non-trainable encoder degrades diffusion loss relative to the VDM baseline, while the trainable encoder matches or improves it.
- The ELBO diverges in continuous time if the generative and forward variances differ, so equal variances are required in the infinite-depth limit.
Experimental setup
Datasets: MNIST CIFAR-10 ImageNet32
Benchmarks: CIFAR-10 ImageNet32 MNIST
Metrics: bits per dimension FID latent loss diffusion loss reconstruction loss p-value from t-test
Baselines: VDM VDMv Flow Matching OT Stochastic Interpolants NVAE VDVAE ARDM Reflected Diffusion Models
Builds on: Variational Diffusion Models DDPM v-prediction Implicit non-linear diffusion models
Closest papers by meaning
- Generalization in VAE and Diffusion Models: A Unified Information-Theoretic Analysis (ICLR 2025 · similarity 0.90)
- Neural Diffusion Models (ICML 2024 · similarity 0.89)
- A Variational Perspective on Solving Inverse Problems with Diffusion Models (ICLR 2024 · similarity 0.89)
- Denoising Diffusion Variational Inference: Diffusion Models as Expressive Variational Posteriors (AAAI 2025 · similarity 0.89)
- Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference (NeurIPS 2024 · similarity 0.89)
- Diffusion Models With Learned Adaptive Noise (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.