Research map / Probabilistic and scientific modeling / Variational and generative sampling
Categorical Reparameterization with Gumbel-Softmax
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace categorical sampling with Gumbel-Softmax reparameterization | SBN with Bernoulli latent variables · MNIST SBN (Bern.) | negative log-likelihood (nats): 58.7 -> 58.5 | Helped |
| Replace categorical sampling with Gumbel-Softmax reparameterization | SBN with categorical latent variables · MNIST SBN (Cat.) | negative log-likelihood (nats): 61.1 -> 59.0 | Helped |
| Replace latent sampling gradients with Gumbel-Softmax reparameterization | VAE with Bernoulli latent variables · MNIST VAE (Bern.) | negative variational lower bound (nats): 109.7 -> 105.0 | Helped |
| Replace latent sampling gradients with Gumbel-Softmax reparameterization | VAE with categorical latent variables · MNIST VAE (Cat.) | negative variational lower bound (nats): 107.0 -> 101.5 | Helped |
| Use single-sample Gumbel inference instead of marginalizing over classes | Semi-supervised VAE · Binarized MNIST, unlabeled test set | accuracy: 92.6% -> 92.4% | No clear effect |
| Use single-sample Gumbel inference instead of marginalizing over classes | Semi-supervised VAE · Binarized MNIST, unlabeled test set | ELBO: -106.8 -> -109.6 | Hurt |
| Use single-sample Gumbel inference instead of marginalizing over classes | Semi-supervised VAE · Semi-supervised VAE, 10 classes | training speedup: 2× | Helped |
| Use single-sample Gumbel inference instead of marginalizing over classes | Semi-supervised VAE · Semi-supervised VAE, 100 classes | training speedup: 9.9× | Helped |
Closest papers by meaning
- Beyond Softmax: A Natural Parameterization for Categorical Random Variables (ICML 2026 · similarity 0.89)
- Categorical Reparameterization with Denoising Diffusion Models (ICML 2026 · similarity 0.88)
- Gradient Estimation with Stochastic Softmax Tricks (NeurIPS 2020 · similarity 0.88)
- Leveraging Recursive Gumbel-Max Trick for Approximate Inference in Combinatorial Spaces (NeurIPS 2021 · similarity 0.87)
- Rao-Blackwellizing the Straight-Through Gumbel-Softmax Gradient Estimator (ICLR 2021 · similarity 0.87)
- Generative Marginalization Models (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.