Research map / Learning theory and algorithms / Kernels and regression theory
Strong inductive biases provably prevent harmless interpolation
Michael Aerni, Marco Milanta, Konstantin Donhauser, Fanny Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a small CNN filter size rather than a large one | CNNs on synthetic images with noisy labels · Synthetic images, 20% label noise | test error gap between interpolation and optimal early stopping | Hurt |
| Increase CNN filter size | CNNs on synthetic images with noisy labels · Synthetic images, 20% label noise | test error gap between interpolation and optimal early stopping | Improved |
| Interpolate instead of optimally early-stop | CNNs on synthetic images with noiseless labels · Synthetic images, noiseless | test error | No clear effect |
| Use a small filter size and optimal early stopping | CNNs on synthetic images with 20% label noise · Noisy training subset | training 0-1 error | Improved |
| Use a large filter size and optimal early stopping | CNNs on synthetic images with 20% label noise · Noisy training subset | training 0-1 error | Hurt |
| Increase rotational-invariance augmentation | Wide ResNets on EuroSAT with noisy labels · EuroSAT, 20% label noise | test error gap between interpolation and optimal early stopping | Hurt |
| Reduce the number of augmented rotations | Wide ResNets on EuroSAT with noisy labels · EuroSAT, 20% label noise | test error gap between interpolation and optimal early stopping | Improved |
| Use rotational invariance and optimal early stopping | Wide ResNets on EuroSAT with 20% label noise · Noisy training subset | training 0-1 error | Improved |
| Use no rotational invariance and optimal early stopping | Wide ResNets on EuroSAT with 20% label noise · Noisy training subset | training 0-1 error | Hurt |
| Interpolate rather than use optimal ridge regularization | Convolutional-kernel regression with a strong inductive bias (small filter size) · High-dimensional kernel regression with noisy data | population risk | Hurt |
| Interpolate rather than use optimal ridge regularization | Convolutional-kernel regression with a weak inductive bias (large filter size) · High-dimensional kernel regression with noisy data | population risk | No clear effect |
| Use a weak inductive bias and optimally tune ridge regularization | Convolutional-kernel regression · High-dimensional kernel regression with noisy data | training error relative to noise level | Improved |
Closest papers by meaning
- Noisy Interpolation Learning with Shallow Univariate ReLU Networks (ICLR 2024 · similarity 0.87)
- Benign Overfitting in Adversarial Training of Neural Networks (ICML 2024 · similarity 0.87)
- Beyond Benign Overfitting in Nadaraya-Watson Interpolators (NeurIPS 2025 · similarity 0.86)
- Rethinking Benign Overfitting in Two-Layer Neural Networks (ICML 2025 · similarity 0.86)
- Position: Deep Learning is Not So Mysterious or Different (ICML 2025 · similarity 0.85)
- Minimum-Norm Interpolation Under Covariate Shift (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
