Research map / Learning theory and algorithms / Kernels and regression theory
Information-theoretic generalization bounds for black-box learning algorithms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Measure information in predictions rather than model weights | Generalization bounds for neural networks · CIFAR-10, ResNet-50 fine-tuning | generalization bound: 5% generalization gap; 16% f-CMI bound | Mixed |
| Measure information in predictions rather than weights | ResNet-50 fine-tuned with SGLD · CIFAR-10 subset (n=20000), after one epoch | generalization bound: approximately 0.45 -> approximately 0.1 | Helped |
| Use the SGLD bound instead of the f-CMI bound | ResNet-50 fine-tuned with SGLD · CIFAR-10, through epoch 12 | generalization bound | Mixed |
| Use the f-CMI bound rather than the SGLD bound | ResNet-50 fine-tuned with SGLD · CIFAR-10, after epoch 16 | tracking of generalization gap | Helped |
| Estimate the bound in early training epochs | f-CMI bound · CIFAR-10 SGLD and MNIST 4 vs 9 deterministic CNN | tightness to generalization gap | Hurt |
| Apply prediction-based f-CMI to deterministic training | CNN on MNIST 4 vs 9 · MNIST 4 vs 9 | informativeness of generalization bound: 75 training examples | Helped |
Closest papers by meaning
- Towards Generalization beyond Pointwise Learning: A Unified Information-theoretic Perspective (ICML 2024 · similarity 0.91)
- Exactly Tight Information-theoretic Generalization Bounds via Binary Jensen-Shannon Divergence (ICML 2025 · similarity 0.90)
- Tighter CMI-Based Generalization Bounds via Stochastic Projection and Quantization (NeurIPS 2025 · similarity 0.90)
- Slicing Mutual Information Generalization Bounds for Neural Networks (ICML 2024 · similarity 0.90)
- Fantastic Generalization Measures are Nowhere to be Found (ICLR 2024 · similarity 0.89)
- Generalizability of Neural Networks Minimizing Empirical Risk Based on Expressive Power (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.