Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Experimental Design for Overparameterized Learning With Application to Single Shot Deep Active Learning
Neta Shoham, Haim Avron
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use bias-oriented criterion t=0 instead of t=0.5 (ablation) | greedy overparameterized design · MNIST kernel regression (lambda=0) | MSE | Improved |
| use t=0 instead of t=lambda (ablation) | greedy overparameterized design · MNIST kernel regression (lambda=0.75^2) | MSE | Improved |
| use t=0 instead of t=lambda at lambda=0.001 (ablation) | kernel ridge regression design · UCI (112 datasets) | datasets with better test error: 5 vs 8, 99 same | No clear effect |
| use t=0 instead of t=lambda at lambda=1.0 (ablation) | kernel ridge regression design · UCI (112 datasets) | datasets with better test error: 21 vs 43, 48 same | Improved |
| use t=0 instead of t=lambda at lambda=10.0 (ablation) | kernel ridge regression design · UCI (112 datasets) | datasets with better test error: 19 vs 68, 25 same | Improved |
| select training set with proposed NTK-based design instead of random | Wide-LeNet5 on MNIST · MNIST single shot | accuracy: random needs 600 samples to match 400 | Improved |
| select training set with proposed design instead of random on narrow LeNet5 | LeNet5 on MNIST · MNIST single shot | accuracy: random needs 600 samples to match 400 | Improved |
| use wide network instead of narrow LeNet5 (ablation) | proposed design and random design · MNIST single shot | accuracy | Improved |
| use sequential uncertainty sampling instead of single shot design with larger training sets | LeNet5 on MNIST · MNIST (LeNet5, batch 100) | accuracy | Mixed |
| use overparameterized OED instead of classical variance-oriented OED | minimum norm linear regression · synthetic linear regression (d=100) | test MSE | Improved |
Closest papers by meaning
- Towards Sample-efficient Overparameterized Meta-learning (NeurIPS 2021 · similarity 0.86)
- Optimal Sample Selection Through Uncertainty Estimation and Its Application in Deep Learning (JMLR 2025 · similarity 0.86)
- More is Better: when Infinite Overparameterization is Optimal and Overfitting is Obligatory (ICLR 2024 · similarity 0.86)
- Bias of Stochastic Gradient Descent or the Architecture: Disentangling the Effects of Overparameterization of Neural Networks (ICML 2024 · similarity 0.84)
- Incorporating Surrogate Gradient Norm to Improve Offline Optimization Techniques (NeurIPS 2024 · similarity 0.84)
- Tilting the Odds at the Lottery: the Interplay of Overparameterisation and Curricula in Neural Networks (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
