Research map / Probabilistic and scientific modeling / Variational and generative sampling
The continuous Bernoulli: fixing a pervasive error in variational autoencoders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| include the continuous Bernoulli normalizing constant instead of using the unnormalized Bernoulli objective | VAE · MNIST | ELBO: 300 nat improvement | Helped |
| include the continuous Bernoulli normalizing constant | VAE · warped MNIST across γ settings | inception score | Helped |
| include the continuous Bernoulli normalizing constant | VAE · warped MNIST across γ settings | 15-nearest-neighbor accuracy | Helped |
| apply the μ⁻¹ parameter correction after training a Bernoulli VAE (ablation) | Bernoulli VAE · warped MNIST | ELBO | Helped |
| apply the μ⁻¹ parameter correction after training a Bernoulli VAE (ablation) | Bernoulli VAE · warped MNIST | ELBO | Hurt |
| include the continuous Bernoulli normalizing constant rather than omit it (ablation) | VAE · CIFAR-10 | ELBO: 475 -> 1007 | Helped |
| include the continuous Bernoulli normalizing constant rather than omit it (ablation) | VAE · CIFAR-10 | IS w/ parameters: 1.39 -> 2.31 | Helped |
| include the Gaussian normalizing constant (ablation) | Gaussian VAE · CIFAR-10 | ELBO: -42411 -> 1891 | Helped |
| include the beta normalizing constant (ablation) | Beta VAE · CIFAR-10 | ELBO: -38913 -> 3121 | Helped |
| use a beta likelihood instead of a continuous Bernoulli likelihood | VAE · CIFAR-10 | ELBO: 1007 -> 3121 | Helped |
| use a beta likelihood instead of a continuous Bernoulli likelihood | VAE · CIFAR-10 | IS w/ samples: 1.15 -> 2.98 | Helped |
| use a beta likelihood instead of a continuous Bernoulli likelihood | VAE · CIFAR-10 | 15-nearest-neighbor accuracy: 0.43 -> 0.47 | Helped |
Closest papers by meaning
- On the Value of Infinite Gradients in Variational Autoencoder Models (NeurIPS 2021 · similarity 0.90)
- Marginalization is not Marginal: No Bad VAE Local Minima when Learning Optimal Sparse Representations (ICML 2023 · similarity 0.88)
- A Statistical Analysis of Wasserstein Autoencoders for Intrinsically Low-dimensional Data (ICLR 2024 · similarity 0.88)
- Variational Laplace Autoencoders (ICML 2019 · similarity 0.88)
- Gaussian Process Modeling of Approximate Inference Errors for Variational Autoencoders (CVPR 2022 · similarity 0.88)
- Sparse Autoencoders, Again? (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.