Research map / Learning theory and algorithms / Sample complexity
CLUB: A Contrastive Log-ratio Upper Bound of Mutual Information
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CLUB instead of L1Out for MI estimation | MI estimator · Gaussian MI estimation, true MI 6 | MSE: 0.75 -> 0.70 | Helped |
| Use CLUB instead of L1Out for MI estimation | MI estimator · Cubic MI estimation, true MI 4 | MSE: 5.58 -> 5.89 | Hurt |
| Add negative sampling to CLUB (ablation) | CLUB · Permutation-invariant MNIST | misclassification rate (%): 1.12 -> 1.10 | Helped |
| Add a variational conditional approximation (ablation) | CLUB · Permutation-invariant MNIST | misclassification rate (%): 1.12 -> 1.10 | Helped |
| Add negative sampling to vCLUB (ablation) | vCLUB · Permutation-invariant MNIST | misclassification rate (%): 1.10 -> 1.06 | Helped |
| Add negative sampling to CLUB (ablation) | CLUB · MNIST to MNIST-M | target accuracy: 93.7 -> 94.6 | Helped |
| Add negative sampling to CLUB (ablation) | CLUB · SVHN to MNIST | target accuracy: 89.7 -> 90.6 | Helped |
| Use negative sampling rather than averaging all negative pairs (ablation) | CLUB · MI estimation, Gaussian and Cubic setups | MSE: 0.15 -> 0.38; 0.12 -> 0.44; 0.70 -> 1.31; 4.53 -> 5.30; 16.57 -> 17.63; 2.22 -> 2.37; 5.89 -> 5.89; 8.25 -> 8.07; 8.23 - | Mixed |
| Use CLUB-S instead of CLUB (ablation) | MI-based UDA framework · UDA transfer tasks | target accuracy: 93.7 -> 94.6; 98.9 -> 98.9; 97.7 -> 98.1; 89.7 -> 90.6; 78.7 -> 79.1; 71.8 -> 72.3 | Mixed |
| Use CLUB-S instead of MCD | UDA framework · SVHN to MNIST | target accuracy: 92.6 -> 90.6 | Hurt |
Closest papers by meaning
- Connecting Jensen–Shannon and Kullback–Leibler Divergences: A New Bound for Representation Learning (NeurIPS 2025 · similarity 0.86)
- Mutual Information Estimation via $f$-Divergence and Data Derangements (NeurIPS 2024 · similarity 0.86)
- Neural Mutual Information Estimation with Vector Copulas (NeurIPS 2025 · similarity 0.85)
- A Benchmark Suite for Evaluating Neural Mutual Information Estimators on Unstructured Datasets (NeurIPS 2024 · similarity 0.84)
- Algorithms and SQ Lower Bounds for Robustly Learning Real-valued Multi-Index Models (NeurIPS 2025 · similarity 0.84)
- Exactly Tight Information-theoretic Generalization Bounds via Binary Jensen-Shannon Divergence (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.