Research map / Probabilistic and scientific modeling / Variational and generative sampling
Consistency Regularization for Variational Auto-Encoders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add KL consistency regularization (lambda=0.1) on top of augmentation (ablation) | VAE · MNIST | negative log-likelihood: 82.8 -> 81.2 | Helped |
| add augmentation-only training (lambda=0) to VAE (ablation) | VAE · MNIST | active units: 36 -> 42 | Helped |
| add KL consistency term on top of augmentation (ablation) | VAE · CIFAR-10 | linear-probe accuracy (%): 40.1 -> 44.7 | Helped |
| add consistency regularization to IWAE | IWAE · MNIST | negative log-likelihood: 81.7 -> 79.7 | Helped |
| add consistency regularization to beta-VAE (0.5) | beta-VAE · MNIST | negative log-likelihood: 92.6 -> 85.7 | Helped |
| add consistency regularization to beta-VAE (10) | beta-VAE (10) · MNIST | negative log-likelihood: 126.1 -> 126.2 | No clear effect |
| add consistency regularization to NVAE | NVAE · CIFAR-10 | bits per dim: 2.91 -> 2.51 | Helped |
| add consistency regularization to NVAE beyond augmentation (ablation) | NVAE · CIFAR-10 | linear-probe accuracy (%): 66.4 -> 71.4 | Helped |
| increase lambda from 0.1 to 1 (ablation) | CR-VAE · MNIST | negative log-likelihood: 81.2 -> 83.9 | Hurt |
| use small lambda (0.001) instead of 0.1 (ablation) | CR-VAE · MNIST | negative log-likelihood: 83.7 -> 83.5 | Helped |
| use all three augmentations instead of scaling only (ablation) | CR-VAE · MNIST | negative log-likelihood: 82.7 -> 81.2 | Helped |
| replace triplet loss with consistency regularization | VAE · MNIST | negative log-likelihood: 83.1 -> 81.2 | Helped |
Closest papers by meaning
- MAE: Mutual Posterior-Divergence Regularization for Variational AutoEncoders (ICLR 2019 · similarity 0.93)
- The Autoencoding Variational Autoencoder (NeurIPS 2020 · similarity 0.92)
- Variational Autoencoder with Implicit Optimal Priors (AAAI 2019 · similarity 0.92)
- Improving Variational Autoencoders with Density Gap-based Regularization (NeurIPS 2022 · similarity 0.92)
- Trading Information between Latents in Hierarchical Variational Autoencoders (ICLR 2023 · similarity 0.92)
- On the Value of Infinite Gradients in Variational Autoencoder Models (NeurIPS 2021 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.