Research map / Probabilistic and scientific modeling / Variational and generative sampling
Semi-Amortized Variational Autoencoders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| initialize encoder-predicted variational parameters and refine them with differentiable SVI, training through the refinement steps | VAE · Yahoo questions | NLL: 330.2 -> 327.5 | Helped |
| initialize encoder-predicted variational parameters and refine them with differentiable SVI | VAE · Yahoo questions | perplexity: 62.5 -> 60.4 | Helped |
| use end-to-end training through SVI refinement instead of VAE + SVI updates that train the encoder from the initial variational parameters | VAE + SVI · Yahoo questions | NLL: 330.5 -> 327.5 | Helped |
| use end-to-end training through SVI refinement instead of training the encoder with the KL distance to the refined posterior | VAE + SVI + KL · Yahoo questions | NLL: 330.1 -> 327.5 | Helped |
| initialize SVI from encoder parameters rather than random initialization | SVI · OMNIGLOT | NLL: 90.27 -> 90.05 | Helped |
| increase inference steps from K=10 to K=20 (ablation) | SA-VAE · Yahoo questions | NLL: 327.6 -> 327.5 | Helped |
| increase inference steps from K=10 to K=20 (ablation) | SA-VAE · OMNIGLOT | NLL: 90.20 -> 90.05 | Helped |
| use a 2-layer MLP inference network instead of a 3-layer ResNet (ablation) | SA-VAE · OMNIGLOT, 100% training data, 12-layer PixelCNN | NLL: 90.05 -> 90.25 | Hurt |
| vary training-set size from 25% to 100% (ablation) | SA-VAE · OMNIGLOT | NLL: 91.89; 91.01; 90.51; 90.05 | Mixed |
| increase PixelCNN depth from 1 to 12 layers (ablation) | SA-VAE · OMNIGLOT | NLL: 96.01 -> 93.16 -> 90.79 -> 90.28 -> 90.05 | Helped |
| pretrain the encoder with a language model (ablation) | VAE · Yahoo questions | NLL: 330.2 -> 330.5 | No clear effect |
| add word dropout at 25% (ablation) | VAE · Yahoo questions | NLL: 330.2 -> 334.2 | Hurt |
Closest papers by meaning
- Amortized Inference Regularization (NeurIPS 2018 · similarity 0.92)
- Gaussian Process Modeling of Approximate Inference Errors for Variational Autoencoders (CVPR 2022 · similarity 0.92)
- InfoVAE: Balancing Learning and Inference in Variational Autoencoders (AAAI 2019 · similarity 0.92)
- Inference Suboptimality in Variational Autoencoders (ICML 2018 · similarity 0.91)
- Variational Laplace Autoencoders (ICML 2019 · similarity 0.91)
- Iterative Amortized Inference (ICML 2018 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.