Research map / Label-efficient and robust learning / Recognition and long-tail
Learning deep representations by mutual information estimation and maximization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use local rather than global-only mutual-information maximization (ablation) | DIM · CIFAR10 | top-1 classification accuracy (Y): 43.17 -> 69.13 | Helped |
| use local rather than global-only mutual-information maximization (ablation) | DIM · CIFAR100 | top-1 classification accuracy (Y): 19.98 -> 41.61 | Helped |
| use local rather than global-only mutual-information maximization (ablation) | DIM · Tiny ImageNet | top-1 classification accuracy (Y): 4.95 -> 33.33 | Helped |
| use local rather than global-only mutual-information maximization (ablation) | DIM · STL-10 | top-1 classification accuracy (Y): 28.09 -> 67.08 | Helped |
| use infoNCE rather than JSD as the MI estimator (ablation) | DIM(L) · CIFAR10 | top-1 classification accuracy (Y): 66.96 -> 69.13 | Helped |
| use infoNCE rather than JSD as the MI estimator (ablation) | DIM(L) · STL-10 | top-1 classification accuracy (Y): 65.93; 67.08 | Mixed |
| add coordinate prediction (ablation) | DIM · CIFAR10 | classification accuracy (Y): 70.65 -> 71.56 | Helped |
| add input occlusion when computing global features (ablation) | DIM · CIFAR10 | classification accuracy (Y): 70.65 -> 72.87 | Helped |
| add coordinate prediction and input occlusion (ablation) | DIM · CIFAR100 | classification accuracy (Y): 44.27 -> 45.96 | Helped |
| use a 3x3-block global representation instead of one global feature (ablation) | DIM(L) · CIFAR10 (no data augmentation) | classification accuracy: 80.95 -> 77.51 | Hurt |
| use a 3x3-block global representation instead of one global feature (ablation) | DIM(L) · STL-10 (random crop pretraining) | classification accuracy: 76.97 -> 78.21 | Helped |
| add a small global MI term to the local objective (ablation) | DIM(L) · CIFAR10 | SVM accuracy (Y): 49.62 -> 18.63 | Hurt |
Closest papers by meaning
- An Information-Theoretic Framework for Fast and Robust Unsupervised Learning via Neural Population Infomax (ICLR 2017 · similarity 0.90)
- Efficient Distribution Matching of Representations via Noise-Injected Deep InfoMax (ICLR 2025 · similarity 0.90)
- On Mutual Information Maximization for Representation Learning (ICLR 2020 · similarity 0.90)
- Mutual Information Gradient Estimation for Representation Learning (ICLR 2020 · similarity 0.86)
- Learning Discrete Structured Representations by Adversarially Maximizing Mutual Information (ICML 2020 · similarity 0.86)
- A Global Geometric Analysis of Maximal Coding Rate Reduction (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.