Research map / Probabilistic and scientific modeling / Variational and generative sampling
Mutual Information Neural Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Maximize mutual information between generated samples and latent code (MINE regularizer added to GAN objective) | GAN · Stacked MNIST | Modes captured (max 1000) | Helped |
| Maximize mutual information between generated samples and latent code | GAN · Stacked MNIST | KL divergence | Helped |
| Maximize mutual information between x and z in ALI | ALI · MNIST | Reconstruction error: 14.24 -> 9.73 | Helped |
| Maximize mutual information between x and z in ALI | ALI · CelebA | Reconstruction accuracy (%): 57.49 -> 76.08 | Helped |
| Maximize mutual information between x and z in ALI | ALI · CelebA | Reconstruction error: 53.75 -> 36.11 | Helped |
| Use MINE as the regularizer for the information bottleneck with additive-noise encoder | Information bottleneck classifier on permutation-invariant MNIST · Permutation-invariant MNIST | Misclassification rate (%): 1.38 baseline -> 1.01 | Helped |
| Replace DVB variational bound with MINE estimator (Gaussian encoder) (ablation) | Information bottleneck on permutation-invariant MNIST · Permutation-invariant MNIST | Misclassification rate (%): 1.13 (DVB) -> 1.11 | Helped |
| Use MINE-based estimator with noise in encoder input (additive vs Gaussian encoder) (ablation) | Information bottleneck on permutation-invariant MNIST · Permutation-invariant MNIST | Misclassification rate (%) | Mixed |
| Use Donsker-Varadhan bound (MINE) instead of f-divergence bound (MINE-f) (ablation) | Mutual information estimation between Gaussians · Multivariate Gaussian, 20 dimensions | Estimate tightness vs ground truth | Helped |
| Use MINE instead of k-NN estimator (Kraskov) | Mutual information estimation between Gaussians · Multivariate Gaussian, 20 dimensions | Estimate accuracy vs ground truth | Helped |
| Use of bias-corrected gradient (moving average in denominator) in MINE (ablation) | MINE training · Mutual information estimation | Overall performance | Helped |
Closest papers by meaning
- InfoNet: Neural Estimation of Mutual Information without Test-Time Optimization (ICML 2024 · similarity 0.91)
- Mutual Information Estimation via $f$-Divergence and Data Derangements (NeurIPS 2024 · similarity 0.89)
- Neural Mutual Information Estimation with Vector Copulas (NeurIPS 2025 · similarity 0.88)
- A Benchmark Suite for Evaluating Neural Mutual Information Estimators on Unstructured Datasets (NeurIPS 2024 · similarity 0.88)
- MINDE: Mutual Information Neural Diffusion Estimation (ICLR 2024 · similarity 0.86)
- Mutual Information Estimation via Normalizing Flows (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.