Research map / Label-efficient and robust learning / Recognition and long-tail
Learning Optimal Representations with the Decodable Information Bottleneck
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DIB representations rather than the strongest reported alternative, V^-‑DIB | Downstream worst ERM on CIFAR10 · CIFAR10 | log loss: 1.54 {\scriptstyle \,\pm\, .03} -> 1.41 {\scriptstyle \,\pm\, .01} | Helped |
| Use DIB representations rather than the strongest reported alternative, V^-‑DIB | Downstream average ERM on CIFAR10 · CIFAR10 | log loss: 1.47 {\scriptstyle \,\pm\, .01} -> 1.38 {\scriptstyle \,\pm\, .01} | Helped |
| Use DIB instead of VIB | Downstream worst ERM · CIFAR10 | log loss: 1.82 {\scriptstyle \,\pm\, .02} -> 1.41 {\scriptstyle \,\pm\, .01} | Helped |
| Increase the DIB beta penalty on V-minimality (ablation) | DIB representation and downstream worst ERM · CIFAR10 | train-test generalization gap | Helped |
| Increase beta past the reported threshold, reducing V-sufficiency (ablation) | DIB representation and downstream classifier · CIFAR10 | best achievable loss: beta=10 | Hurt |
| Increase beta to suppress decodable distractor information (ablation) | DIB representation and downstream ERM · CIFAR10 with overlaid MNIST distractors | generalization | Helped |
| Use a larger predictor family for the minimality measurement (ablation) | V_L generalization probe · 562 CNNs | Kendall rank correlation with accuracy generalization gap: 0.471 -> 0.482 | Helped |
| Use a larger predictor family for the minimality measurement (ablation) | V_L generalization probe · 562 CNNs | Kendall rank correlation with log-loss generalization gap: 0.498 -> 0.505 | Helped |
Closest papers by meaning
- Minimum Description Length and Generalization Guarantees for Representation Learning (NeurIPS 2023 · similarity 0.89)
- Learning Robust Representations via Multi-View Information Bottleneck (ICLR 2020 · similarity 0.88)
- Information Bottleneck Analysis of Deep Neural Networks via Lossy Compression (ICLR 2024 · similarity 0.87)
- Does Compression Imply Generalization? A Minimum Description Length Perspective (NeurIPS 2026 · similarity 0.86)
- Cauchy-Schwarz Divergence Information Bottleneck for Regression (ICLR 2024 · similarity 0.86)
- Structured IB: Improving Information Bottleneck with Structured Feature Learning (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.