Research map / Probabilistic and scientific modeling / Variational and generative sampling
Exact Rate-Distortion in Autoencoders via Echo Noise
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Echo noise with exact mutual-information rate regularization instead of the strongest listed comparison | Autoencoder · Binary MNIST test | -ELBO: 89.2 -> 88.8 | Helped |
| Use Echo noise with exact mutual-information rate regularization instead of the strongest listed comparison | Autoencoder · Omniglot test | -ELBO: 115.1 -> 114.6 | Helped |
| Use Echo noise with exact mutual-information rate regularization instead of the strongest listed comparison | Autoencoder · Fashion MNIST test | -ELBO: 234.3 -> 234.9 | Hurt |
| Add an MMD penalty to encourage the encoding marginal to match the Gaussian prior (ablation) | InfoVAE · test log-likelihood bounds across image datasets | -ELBO | Helped |
| Add an MMD penalty to encourage the encoding marginal to match the Gaussian prior (ablation) | IAF encoder models · test log-likelihood bounds across image datasets | -ELBO | No clear effect |
| Add independence regularization (ablation) | Echo · dSprites with independent ground-truth factors | Mutual Information Gap | Helped |
| Add independence regularization (ablation) | VAE · dSprites with independent ground-truth factors | disentanglement score | Helped |
| Add independence regularization when the ground-truth factors are dependent (ablation) | Echo and VAE · dSprites with dependent ground-truth factors | disentanglement score | Hurt |
| Increase beta (ablation) | Echo and VAE · dSprites with dependent ground-truth factors | disentanglement score | Hurt |
Closest papers by meaning
- Improving Inference for Neural Image Compression (NeurIPS 2020 · similarity 0.88)
- Compression of Structured Data with Autoencoders: Provable Benefit of Nonlinearities and Depth (ICML 2024 · similarity 0.87)
- Universally Quantized Neural Compression (NeurIPS 2020 · similarity 0.87)
- Variational image compression with a scale hyperprior (ICLR 2018 · similarity 0.87)
- Hierarchical Quantized Autoencoders (NeurIPS 2020 · similarity 0.87)
- Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.