Research map / Learning theory and algorithms / Kernels and regression theory
Sketched Ridgeless Linear Regression: The Role of Downsampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Downsample using orthogonal sketching rather than i.i.d. sketching | Ridgeless linear regression · Underparameterized regime, including isotropic and correlated features | limiting out-of-sample prediction risk: \dfrac{\sigma^2\phi}{1-\phi} + \dfrac{\sigma^2\phi\psi^{-1}}{1-\phi\psi^{-1}} -> \dfrac{\sigma^2\phi\psi^{-1}}{1-\phi\ps | Helped |
| Use orthogonal instead of i.i.d. sketching | Ridgeless linear regression · Overparameterized regime (φψ⁻¹ > 1) | limiting out-of-sample prediction risk | No clear effect |
| Select a nontrivial sketching size in the stated SNR and aspect-ratio range | Ridgeless linear regression with isotropic features · SNR > 1 and φ ∈ (1-σ/(2α), α/(α-σ)] | limiting out-of-sample prediction risk: m^*=\frac{\alpha-\sigma}{\alpha}\phi \cdot n | Helped |
| Use the null estimator instead of fitting a sketched ridgeless estimator | Ridgeless linear regression with isotropic features · SNR ≤ 1 and φ > α²/(α²+σ²) | limiting out-of-sample prediction risk: m^*=0 | Helped |
| Tune sketching size using a validation dataset | Orthogonally sketched ridgeless regression · Correlated features | out-of-sample prediction risk stability: n_{\text{val}}=20 | Helped |
| Sketch the estimator into the underparameterized regime | Ridgeless regression under the described misspecified model · Misspecified model | limiting out-of-sample prediction risk | No clear effect |
Closest papers by meaning
- Asymptotically Free Sketched Ridge Ensembles: Risks, Cross-Validation, and Tuning (ICLR 2024 · similarity 0.90)
- Prediction Risk and Estimation Risk of the Ridgeless Least Squares Estimator under General Assumptions on Regression Errors (ICLR 2025 · similarity 0.87)
- Risk and cross validation in ridge regression with correlated samples (ICML 2025 · similarity 0.86)
- Optimal Ridge Regularization for Out-of-Distribution Prediction (ICML 2024 · similarity 0.86)
- Distributed Least Squares in Small Space via Sketching and Bias Reduction (NeurIPS 2024 · similarity 0.85)
- Overfitting Behaviour of Gaussian Kernel Ridgeless Regression: Varying Bandwidth or Dimensionality (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.