Research map / Probabilistic and scientific modeling / Variational and generative sampling
The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables
Chris J. Maddison, Andriy Mnih, Yee Whye Teh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Concrete relaxations instead of VIMCO for nonlinear density models | Discrete latent-variable density estimation · MNIST and Omniglot | test NLL: 92.1 vs 93.8; 108.4 vs 116.4 | Improved |
| Use Concrete relaxations instead of VIMCO for nonlinear density models | Discrete latent-variable density estimation · MNIST and Omniglot | test NLL: 87.9 vs 88.4; 105.9 vs 111.7 | Improved |
| Use Concrete relaxations instead of VIMCO for nonlinear density models | Discrete latent-variable density estimation · MNIST and Omniglot | test NLL: 85.7 vs 85.5; 106.8 vs 113.2 | Mixed |
| Use Concrete relaxations instead of VIMCO for linear density models | Discrete latent-variable density estimation · MNIST and Omniglot | test NLL: 107.3 vs 104.4; 118.7 vs 115.7 | Hurt |
| Use Concrete relaxations instead of VIMCO for linear density models | Discrete latent-variable density estimation · MNIST and Omniglot | test NLL: 102.1 vs 92.9; 116.3 vs 109.2 | Hurt |
| Increase latent-node arity from 4 to 8 | Concrete density estimation · MNIST | test NLL: 86.7 -> 87.4 | Hurt |
| Increase latent-node arity from 4 to 8 | Concrete structured prediction · MNIST | test NLL: 55.4 -> 54.7 | Improved |
| Replace Concrete relaxations with Logistic Normal relaxations | 4-ary density model · MNIST | test NLL: 86.7 vs 88.7 | Hurt |
| Replace Concrete relaxations with Logistic Normal relaxations | 8-ary density model · MNIST | test NLL: 87.4 vs 89.1 | Hurt |
| Use distinct temperatures for prior and posterior Concrete distributions | Concrete variational training · Density estimation | test NLL | Improved |
| Anneal or automatically tune the temperature during training | Concrete relaxation · Density estimation and structured prediction | test NLL | No clear effect |
Closest papers by meaning
- Implicit Reparameterization Gradients (NeurIPS 2018 · similarity 0.89)
- Reducing Reparameterization Gradient Variance (NeurIPS 2017 · similarity 0.88)
- The Generalized Reparameterization Gradient (NeurIPS 2016 · similarity 0.87)
- Reparameterization Gradient for Non-differentiable Models (NeurIPS 2018 · similarity 0.87)
- Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics (ICLR 2025 · similarity 0.85)
- The Power of Random Features and the Limits of Distribution-Free Gradient Descent (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
