Research map / Optimization / Optimizers
Tight Mutual Information Estimation With Contrastive Fenchel-Legendre Optimization
Qing Guo, Junya Chen, Dong Wang, Yuewei Yang, Xinwei Deng, Jing Huang, Lawrence Carin, Fan Li, Chenyang Tao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FDV (FLO-derived) objective instead of InfoNCE for contrastive pretraining | SimCLR · CIFAR linear probing | sample efficiency (batch size for equal accuracy): FlatCLR-32 vs SimCLR-256 (8x) | Improved |
| share one network with two heads for g(x,y) and u(x,y) rather than separate critics (ablation) | FLO · 20-D Gaussian rho=0.5 | MI estimation learning speed | Improved |
| use deeper critic MLP (2, 3, 6 layers) (ablation) | FLO · Gaussian MI estimation | convergence speed and MI estimate | Mixed |
| replace NWJ/InfoNCE with FLO as MI objective for design optimization | Bayesian optimal experiment design · BOED static and dynamic designs | expected information gain | Improved |
| add FLO-based regularizer to the empirical risk (Meta-FLO) versus MAML | meta-learning · few-shot regression benchmark | adaptation performance | Improved |
| tune alpha in alpha-InfoNCE | alpha-InfoNCE · Gaussian MI estimation | bias-variance | No clear effect |
Closest papers by meaning
- Mutual Information Estimation via $f$-Divergence and Data Derangements (NeurIPS 2024 · similarity 0.89)
- Connecting Jensen–Shannon and Kullback–Leibler Divergences: A New Bound for Representation Learning (NeurIPS 2025 · similarity 0.87)
- Towards Generalization beyond Pointwise Learning: A Unified Information-theoretic Perspective (ICML 2024 · similarity 0.86)
- Exactly Tight Information-theoretic Generalization Bounds via Binary Jensen-Shannon Divergence (ICML 2025 · similarity 0.86)
- FMMI: Flow Matching Mutual Information Estimation (NeurIPS 2026 · similarity 0.85)
- Mutual Information Estimation via Normalizing Flows (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
