Research map / Optimization / Optimizers
Differentiable Annealed Importance Sampling and the Perils of Gradient Noise
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use stochastic rather than full-batch gradients | DAIS · Bayesian linear regression | log marginal likelihood bound gap: \liminf_{K \rightarrow \infty} |\log p(\mathcal{D}) - \mathcal{L}_\text{DAIS}| > 0 | Hurt |
| Use full-batch gradients with equally spaced annealing and η ∼ 1/K^c, c = 1/4 | DAIS · Bayesian linear regression | log marginal likelihood bound gap: \mathcal{O}(1/\sqrt{K}) | Helped |
| Increase the number of intermediate distributions K | DAVI · Dynamically binarized MNIST | test negative log-likelihood | Helped |
| Use DAVI instead of IWAE at the low-compute setting S×K=5 | Dynamically binarized MNIST · Dynamically binarized MNIST, S×K=5 | test negative log-likelihood: 85.43 -> 86.51 | Hurt |
| Use DAVI instead of IWAE at S×K=500, K=10 | Dynamically binarized MNIST · Dynamically binarized MNIST, S×K=500, K=10 | test negative log-likelihood: 83.87 -> 83.62 | Helped |
| Learn the annealing scheme (ablation) | DAVI · Dynamically binarized MNIST | test negative log-likelihood: 86.51 -> 86.49; 84.49 -> 84.42; 84.45 -> 84.39; 85.23 -> 85.00; 83.62 -> 83.56; 83.65 -> 83.69 | Mixed |
| Remove the Metropolis-Hastings correction | DAIS versus AIS · VAE model evaluation | negative log-likelihood | No clear effect |
| Increase compute for annealing-based evaluation rather than using low-budget importance weighting | DAIS, AIS, and HAIS versus IWAE · VAE-, IWAE-, or DAIS-trained models | negative log-likelihood | Mixed |
Closest papers by meaning
- Differentiable Annealed Importance Sampling Minimizes The Jensen-Shannon Divergence Between Initial and Target Distribution (ICML 2024 · similarity 0.92)
- Score-Based Diffusion meets Annealed Importance Sampling (NeurIPS 2022 · similarity 0.91)
- Adaptive Annealed Importance Sampling with Constant Rate Progress (ICML 2023 · similarity 0.89)
- Forward $\chi^2$ Divergence Based Variational Importance Sampling (ICLR 2024 · similarity 0.87)
- The Polynomial Stein Discrepancy for Assessing Moment Convergence (ICML 2025 · similarity 0.86)
- Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.