Research map / Probabilistic and scientific modeling / Variational and generative sampling
Improving Mutual Information Estimation with Annealed and Energy-Based Bounds
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use energy-based training with MCMC initialized from true posterior samples instead of relying on a GIWAE critic | MINE-AIS · MNIST VAE | MI lower bound: 32% | Helped |
| Use energy-based training with MCMC initialized from true posterior samples instead of relying on a GIWAE critic | MINE-AIS · MNIST GAN | MI lower bound: 50% | Helped |
| Use a learned energy-based posterior with MINE-AIS | MNIST VAE · MNIST VAE | ground-truth MI estimation error: underestimates the ground truth MI by 11% | Mixed |
| Use a learned energy-based posterior with MINE-AIS | MNIST GAN · MNIST GAN | ground-truth MI estimation error: underestimates the ground truth MI by 24% | Mixed |
| Evaluate the IBAL of a GIWAE-trained critic with Multi-Sample AIS rather than the original GIWAE bound | GIWAE critic · MNIST VAE and MNIST GAN | MI lower-bound estimate | No clear effect |
| Use the learned GIWAE critic rather than an optimal importance-weight critic | GIWAE · MNIST VAE | gap from true MI: 31% | Hurt |
| Use the learned GIWAE critic rather than an optimal importance-weight critic | GIWAE · MNIST GAN | gap from true MI: 46% | Hurt |
| Increase the number of intermediate AIS distributions T | Multi-Sample AIS · MNIST and CIFAR generative models | MI bound tightness: linearly | Helped |
| Increase the number of importance samples K for the IWAE MI lower bound | IWAE · MI estimation | improvement over BA lower bound: at most log K | Helped |
| Use the prior p(z) as the proposal rather than a learned q(z|x) | IWAE · MNIST and CIFAR generative models | MI lower bound | Hurt |
| Replace the energy-based IBAL objective with the looser generalized MINE-DV bound | MINE-AIS · High-MI estimation | lower-bound tightness | Hurt |
| Replace the energy-based IBAL objective with generalized MINE-F | MINE-AIS · High-MI estimation | lower-bound tightness | Hurt |
Closest papers by meaning
- Connecting Jensen–Shannon and Kullback–Leibler Divergences: A New Bound for Representation Learning (NeurIPS 2025 · similarity 0.88)
- A Benchmark Suite for Evaluating Neural Mutual Information Estimators on Unstructured Datasets (NeurIPS 2024 · similarity 0.87)
- Mutual Information Estimation via $f$-Divergence and Data Derangements (NeurIPS 2024 · similarity 0.87)
- InfoNet: Neural Estimation of Mutual Information without Test-Time Optimization (ICML 2024 · similarity 0.86)
- Slicing Mutual Information Generalization Bounds for Neural Networks (ICML 2024 · similarity 0.86)
- Exactly Tight Information-theoretic Generalization Bounds via Binary Jensen-Shannon Divergence (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.