Research map / Learning theory and algorithms / Kernels and regression theory
PAC-Bayes Information Bottleneck
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PIB instead of vanilla SGD | VGG · CIFAR10 | test accuracy (%): 77.03(0.57) -> 80.19(0.42) | Helped |
| use PIB instead of vanilla SGD | VGG · CIFAR100 | test accuracy (%): 52.07(0.44) -> 56.47(0.62) | Helped |
| use PIB instead of vanilla SGD | VGG · STL10 | test accuracy (%): 54.31(0.65) -> 58.83(0.75) | Helped |
| use PIB instead of vanilla SGD | VGG · SVHN | test accuracy (%): 93.57(0.67) -> 93.88(0.88) | Helped |
| use ℓ2-norm regularization instead of vanilla SGD | VGG · CIFAR100 | test accuracy (%): 52.07(0.44) -> 50.84(0.71) | Hurt |
| use dropout instead of vanilla SGD | VGG · CIFAR10 | test accuracy (%): 77.03(0.57) -> 78.95(0.60) | Helped |
| increase hidden-layer depth | MLP on MNIST · MNIST | fitting and compression phase speed | Helped |
| increase width beyond 32 hidden units | 2-layer MLP · MNIST | test accuracy | Helped |
| set batch size to 16 | MLP · MNIST | test accuracy and generalization gap | Helped |
| train with PIB regularization | VGG · CIFAR10, CIFAR100, STL10, and SVHN | information stored in weights (IIW) | Helped |
| increase label-noise ratio | 2-layer MLP · CIFAR10 | test accuracy | Hurt |
Closest papers by meaning
- PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization (NeurIPS 2022 · similarity 0.89)
- Information Bottleneck Analysis of Deep Neural Networks via Lossy Compression (ICLR 2024 · similarity 0.88)
- Generalization Bounds via Meta-Learned Model Representations: PAC-Bayes and Sample Compression Hypernetworks (ICML 2025 · similarity 0.87)
- Does Compression Imply Generalization? A Minimum Description Length Perspective (NeurIPS 2026 · similarity 0.86)
- Cauchy-Schwarz Divergence Information Bottleneck for Regression (ICLR 2024 · similarity 0.85)
- Algorithms and SQ Lower Bounds for Robustly Learning Real-valued Multi-Index Models (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.