Research map / Learning theory and algorithms / Prediction and decision losses
Caveats for information bottleneck in deterministic scenarios
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the linear compression penalty I(X;T) with its square | IB optimization · Information bottleneck curves, including deterministic Y=f(X) | ability to recover distinct points along the IB curve as β varies | Helped |
| Use the standard IB Lagrangian rather than the squared-IB functional | Nonlinear IB · MNIST training | number of distinct IB-curve points recovered | Hurt |
| Use the squared-IB functional rather than the standard IB Lagrangian | Nonlinear IB · MNIST training | range of compression-prediction trade-off explored | Helped |
| Use β=0, omitting compression regularization | Nonlinear IB · MNIST training | prediction | Helped |
| Increase β in the squared-IB functional | Nonlinear IB · MNIST training | number of clusters: 10 clusters (β=0.05); 6 clusters (β=0.2); 3 clusters (β=0.5); 1 cluster (β=2.0) | Helped |
| Optimize only the IB Lagrangian while varying β | Information bottleneck · Deterministic Y=f(X) | recovery of distinct points along the IB curve | Hurt |
| Use the IB curve as the sole criterion for choosing a representation | Information bottleneck · Deterministic Y=f(X) | semantic or perceptual usefulness of compressed representations | No clear effect |
| Require zero prediction error in a multilayer classifier | Successive neural-network layers · Deterministic classification | strict trade-off between compression and prediction | Hurt |
| Use the squared-dIB functional rather than the dIB Lagrangian | Deterministic information bottleneck · Deterministic Y=f(X) | recovery of different points on the dIB curve | Helped |
| Assume Y is approximately rather than exactly a deterministic function of X | Information bottleneck · Joint distributions ε-close to deterministic | severity of the three IB caveats: O(-ε log ε) | Mixed |
Closest papers by meaning
- Information Bottleneck Analysis of Deep Neural Networks via Lossy Compression (ICLR 2024 · similarity 0.88)
- Structured IB: Improving Information Bottleneck with Structured Feature Learning (AAAI 2025 · similarity 0.88)
- Cauchy-Schwarz Divergence Information Bottleneck for Regression (ICLR 2024 · similarity 0.88)
- A Rate-Distortion View of Uncertainty Quantification (ICML 2024 · similarity 0.84)
- Sample-efficient Learning of Concepts with Theoretical Guarantees: from Data to Concepts without Interventions (NeurIPS 2025 · similarity 0.84)
- Understanding and Leveraging the Learning Phases of Neural Networks (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.