Research map / Probabilistic and scientific modeling / Variational and generative sampling
Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Learn observation noise σ² instead of fixing it (ablation) | Deep Gaussian VAE · MNIST | ELBO: -1022.1 ± 5.4 -> -1004.6 ± 1.4 | Helped |
| Learn observation noise σ² instead of fixing it (ablation) | Deep Gaussian VAE · CelebA 64 | ELBO: -50760.3 ± 353.4 -> -50631.2 ± 163.4 | Helped |
| Initialize fixed observation noise at a large value (ablation) | Deep Gaussian VAE · MNIST | posterior collapse (%): 27.38 -> 89.88 | Hurt |
| Initialize fixed observation noise at a large value (ablation) | Deep Gaussian VAE · CelebA 64 | posterior collapse (%): 2.52 -> 80.56 | Hurt |
| Vary the initialization of learnable observation noise (ablation) | Deep Gaussian VAE · MNIST | ELBO: -1004.6 ± 1.4 -> -1038.7 ± 5.6 | Mixed |
| Vary the initialization of learnable observation noise (ablation) | Deep Gaussian VAE · CelebA 64 | posterior collapse (%): 0.00 -> 54.76 | Mixed |
| Use stochastic rather than analytic ELBO gradients (ablation) | Linear VAE · MNIST | training ELBO | Hurt |
| Anneal the KL weight while keeping σ² fixed (ablation) | Deep Gaussian VAE · MNIST | posterior collapse | Mixed |
| Anneal the KL weight while learning σ² (ablation) | Deep Gaussian VAE · CelebA | posterior collapse | Helped |
| Optimize the exact marginal likelihood rather than relying on a powerful decoder | pPCA / linear VAE · Linear latent-variable model | posterior collapse | Mixed |
| Use a diagonal variational covariance in the linear VAE | Linear VAE · pPCA principal components | identifiability | Helped |
Closest papers by meaning
- Beyond Vanilla Variational Autoencoders: Detecting Posterior Collapse in Conditional and Hierarchical Variational Autoencoders (ICLR 2024 · similarity 0.93)
- The Usual Suspects? Reassessing Blame for VAE Posterior Collapse (ICML 2020 · similarity 0.93)
- Lagging Inference Networks and Posterior Collapse in Variational Autoencoders (ICLR 2019 · similarity 0.93)
- Posterior Collapse of a Linear Latent Variable Model (NeurIPS 2022 · similarity 0.91)
- Be a Goldfish: Forgetting Bad Conditioning in Sparse Linear Regression via Variational Autoencoders (ICML 2025 · similarity 0.89)
- Variational Learning of Fractional Posteriors (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.