Research map / Learning theory and algorithms / Kernels and regression theory
Subspace Fitting Meets Regression: The Effects of Supervision and Orthonormality Constraints on Double Descent of Generalization Errors
Yehuda Dar, Paul Mayer, Lorenzo Luzi, Richard Baraniuk
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase the number of input features used for fitting (ablation) | Unsupervised PCA subspace fitting · Synthetic subspace-fitting data | out-of-sample reconstruction error | Improved |
| Increase the estimated subspace rank (ablation) | Unsupervised PCA subspace fitting · Synthetic subspace-fitting data | out-of-sample reconstruction error | Improved |
| Use strict orthonormality constraints instead of unconstrained regression (ablation) | Fully supervised subspace fitting · Synthetic supervised subspace-fitting data | out-of-sample squared error | Mixed |
| Remove the orthonormality constraint (ablation) | Fully supervised subspace fitting · Synthetic supervised subspace-fitting data | out-of-sample squared error | Mixed |
| Relax orthonormality constraints progressively (ablation) | Fully supervised subspace fitting · Synthetic supervised subspace-fitting data | out-of-sample squared error | Improved |
| Increase the number of supervised examples (ablation) | Unconstrained subspace fitting · Synthetic semi-supervised subspace-fitting data | out-of-sample generalization error | Mixed |
| Jointly increase supervision and relax orthonormality constraints (ablation) | Subspace fitting along the supervision-orthonormality plane · Synthetic subspace-fitting data | out-of-sample generalization error | Mixed |
| Increase the number of features in unsupervised fitting (ablation) | Unsupervised PCA subspace fitting · Synthetic subspace-fitting data | out-of-sample reconstruction error | No clear effect |
Closest papers by meaning
- Robust Model Reasoning and Fitting via Dual Sparsity Pursuit (NeurIPS 2023 · similarity 0.88)
- Learning Neural Network Subspaces (ICML 2021 · similarity 0.88)
- Implicit Bias of Projected Subgradient Method Gives Provable Robust Recovery of Subspaces of Unknown Codimension (ICLR 2022 · similarity 0.87)
- Neural Networks Perform Sufficient Dimension Reduction (AAAI 2025 · similarity 0.86)
- A Subspace-Constrained Tyler's Estimator and its Applications to Structure from Motion (CVPR 2024 · similarity 0.85)
- Memory-Efficient LLM Training with Online Subspace Descent (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
