Research map / Learning theory and algorithms / Kernels and regression theory
Optimal Nuisance Function Tuning for Estimating a Doubly Robust Functional under Proportional Asymptotics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes ridge-regression nuisance tuning for estimating the expected conditional covariance under proportional asymptotics, derives debiased INT/NR/DR estimators under two- and three-split sample splitting, and shows variance-optimal ridge penalties differ from prediction-optimal ones.
Problem
Estimating the expected conditional covariance (a doubly robust functional) when the covariate dimension p is proportional to the sample size n, where nuisance functions are inconsistent and standard DML theory fails.
Approach
Debiases three ECC plug-in estimators (integral-based, Newey-Robins, doubly robust) by inverting deterministic random-matrix-theory bias functions of ridge estimators, derives their exact asymptotic variances for Gaussian designs, and shows variance-minimizing ridge penalties differ from prediction-optimal ones.</222>
Key results
- Bias-corrected INT, NR and DR estimators are sqrt(n)-consistent under both two-split and three-split schemes for any ridge penalties.
- Variance-optimal ridge tuning parameters differ from prediction-optimal ones, as shown in simulations with p=1000 (c=2) and n=500 per split.
- In the two-split scheme the variance blows up near lambda=1.48 because the bias-correction constant diverges.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply the proposed asymptotic-bias correction to INT, NR, or DR | ECC estimators with ridge nuisance estimates · ECC, two-split and three-split | consistency rate: √n(θ̂^{A, db}_v − θ₀) = O_p(1) | Helped |
| Choose ridge tuning parameters to minimize ECC asymptotic variance rather than nuisance prediction error | ECC estimators · Proportional asymptotics | asymptotic variance | Helped |
| Use prediction-optimal ridge tuning instead of inference-optimal tuning | ECC estimators · Proportional asymptotics | asymptotic variance | Hurt |
| Set the two-split tuning parameter near λ=1.48 | Debiased ECC estimators · Simulation, two-split | variance: λ=1.48 | Hurt |
Surprising findings
- Prediction-optimal ridge penalties do not minimize the asymptotic variance of the ECC estimator.
- The two-split variance diverges at a finite lambda because the bias-correction constant blows up.
Limitations
- Variance formulas derived under Gaussian covariates or Gaussian errors; universality not established.
- Analysis restricted to ridge regression; other estimators such as Lasso left for future work.
- Two-split variance blows up at certain lambda values due to bias-correction constant divergence.
Experimental setup
Metrics: asymptotic variance bias confidence interval length
Baselines: integral-based plug-in estimator Newey-Robins estimator doubly robust estimator
Builds on: double machine learning Marchenko-Pastur law Newey-Robins plug-in estimator
Closest papers by meaning
- It’s Hard to Be Normal: The Impact of Noise on Structure-agnostic Estimation (NeurIPS 2025 · similarity 0.84)
- Doubly Robust Proximal Causal Learning for Continuous Treatments (ICLR 2024 · similarity 0.84)
- TIC-TAC: A Framework For Improved Covariance Estimation In Deep Heteroscedastic Regression (ICML 2024 · similarity 0.84)
- Optimistic Algorithms for Adaptive Estimation of the Average Treatment Effect (ICML 2025 · similarity 0.84)
- Nonparametric Instrumental Variable Regression through Stochastic Approximate Gradients (NeurIPS 2024 · similarity 0.84)
- Prediction Risk and Estimation Risk of the Ridgeless Least Squares Estimator under General Assumptions on Regression Errors (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.