Research map / Learning theory and algorithms / Kernels and regression theory
Does the Data Induce Capacity Control in Deep Learning?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use data-dependent prior with covariance proportional to FIM at initialization (Method 4) instead of isotropic prior | PAC-Bayes bound optimization on MNIST networks · MNIST FC-600-1 / FC-600-2 / LeNet-5 | PAC-Bayes bound: FC-600-1 0.1590 (Method 2) to 0.1323; LeNet 0.0099 to 0.0092 | Helped |
| set posterior eigenvectors to Hessian eigenvectors (Method 3) versus diagonal posterior covariance (Baseline A) | PAC-Bayes posterior covariance · MNIST FC-600-1 | PAC-Bayes bound: 0.161 to 0.1357 | Helped |
| analytical posterior from Hessian eigenvalues (Method 1) with isotropic-like prior, versus numerical methods | MNIST FC-600-1 · MNIST FC-600-1 | PAC-Bayes bound: 0.3241 vs 0.1357 (Method 3) | Hurt |
| sloppy input spectrum (vs flat, non-sloppy) in synthetic data | student networks trained on synthetic data · synthetic datasets with varying sloppy factor | validation error | Helped |
| training predominantly in stiff FIM subspace: projection of weight change onto top-k FIM eigenvectors | FC networks on MNIST · MNIST | projection overlap | No clear effect |
Closest papers by meaning
- PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization (NeurIPS 2022 · similarity 0.88)
- Investigating the Overlooked Hessian Structure: From CNNs to LLMs (ICML 2025 · similarity 0.86)
- Analyzing Neural Scaling Laws in Two-Layer Networks with Power-Law Data Spectra (ICLR 2025 · similarity 0.86)
- Slicing Mutual Information Generalization Bounds for Neural Networks (ICML 2024 · similarity 0.86)
- Generalization Bounds for Rank-sparse Neural Networks (NeurIPS 2025 · similarity 0.86)
- Do Neural Networks Need Gradient Descent to Generalize? A Theoretical Study (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.