Research map / Probabilistic and scientific modeling / Variational and generative sampling
REBAR: Low-variance, unbiased gradient estimates for discrete latent variable models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use REBAR instead of the strongest listed alternative | MNIST generative SBN, linear 1 layer · MNIST gen., linear 1 layer | training variational lower bound: -111.3 -> -111.6 | Hurt |
| Use REBAR instead of MuProp | MNIST generative SBN, linear 2 layer · MNIST gen., linear 2 layer | training variational lower bound: -99.07 -> -98.8 | Helped |
| Use REBAR instead of MuProp | MNIST generative SBN, nonlinear · MNIST gen., nonlinear | training variational lower bound: -101.5 -> -101.1 | Helped |
| Use REBAR instead of REBAR with fixed temperature 0.1 | Omniglot generative SBN, linear 1 layer · Omniglot gen., linear 1 layer | training variational lower bound: -116.93 -> -116.83 | Helped |
| Use REBAR instead of REBAR with fixed temperature 0.1 | Omniglot generative SBN, linear 2 layer · Omniglot gen., linear 2 layer | training variational lower bound: -109.12 -> -108.99 | Helped |
| Use REBAR instead of MuProp | Omniglot generative SBN, nonlinear · Omniglot gen., nonlinear | training variational lower bound: -109.58 -> -108.72 | Helped |
| Use REBAR instead of MuProp | MNIST structured prediction SBN, linear 1 layer · MNIST struct. pred., linear 1 layer | training variational lower bound: -64.33 -> -65.21 | Hurt |
| Use REBAR instead of MuProp | MNIST structured prediction SBN, linear 2 layer · MNIST struct. pred., linear 2 layer | training variational lower bound: -63.69 -> -61.72 | Helped |
| Use REBAR instead of MuProp | MNIST structured prediction SBN, nonlinear · MNIST struct. pred., nonlinear | training variational lower bound: -47.6 -> -46.44 | Helped |
| Replace the discrete objective gradient with the biased Concrete estimator | Toy Bernoulli optimization problem · Toy problem, t = 0.45 | loss | Hurt |
| Adapt the relaxation temperature online instead of fixing it | REBAR · MNIST and Omniglot generative modeling | gradient variance | Helped |
| Use the fixed-temperature Concrete estimator | Nonlinear SBN training · MNIST and Omniglot generative modeling | training variational lower bound | Hurt |
Closest papers by meaning
- Coupled Gradient Estimators for Discrete Latent Variables (NeurIPS 2021 · similarity 0.89)
- Gradient Estimation for Binary Latent Variables via Gradient Variance Clipping (AAAI 2023 · similarity 0.89)
- Gradient Estimation with Discrete Stein Operators (NeurIPS 2022 · similarity 0.88)
- DisARM: An Antithetic Gradient Estimator for Binary Latent Variables (NeurIPS 2020 · similarity 0.87)
- VarGrad: A Low-Variance Gradient Estimator for Variational Inference (NeurIPS 2020 · similarity 0.87)
- Multivariate Latent Recalibration for Conditional Normalizing Flows (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.