Research map / Optimization / Optimizers
Nonparametric Instrumental Variable Regression through Stochastic Approximate Gradients
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAGD-IV, a functional stochastic gradient descent method that minimizes the populational NPIV risk using a density-ratio-based approximate gradient, with excess-risk bounds and support for non-quadratic losses such as binary outcomes.
Problem
Nonparametric instrumental variable (NPIV) regression, an ill-posed inverse problem, where existing methods minimize empirical risks or moment-based minimax objectives and struggle with stability and with binary outcomes.
Approach
Derives an analytical functional gradient of the populational projected risk, replaces the adjoint conditional expectation operator with an estimated density ratio Phi(x,z) so that only Z samples are needed in the inner loop, and runs projected SGD in function space; the loss can be any convex, smooth loss (e.g., binary cross-entropy with a known logistic error CDF).
Key results
- Kernel SAGD-IV had the smallest variance across scenarios among kernel methods, while DualIV showed the highest variance.
- TSLS dominates the linear response case but performs poorly on step, absolute value and sin responses.
- Kernel SAGD-IV performs remarkably well on binary response DGP for sin and linear response functions, with results on par with the continuous case.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SAGD-IV instead of linear 2SLS | Continuous NPIV regression · Continuous response: step, absolute, and sine | log MSE | Helped |
| Use 2SLS instead of nonlinear methods | Continuous NPIV regression · Continuous response: linear | log MSE | Helped |
| Use KIV as a competitor to SAGD-IV | Continuous NPIV regression · Continuous response: step and sine | log MSE | Mixed |
| Use KIV instead of SAGD-IV | Continuous NPIV regression · Continuous response: absolute | log MSE | Hurt |
| Use Deep SAGD-IV instead of the other neural-network-based algorithms | Continuous NPIV regression · Continuous response | log MSE | Helped |
| Use Kernel SAGD-IV among kernel methods | Continuous NPIV regression · Continuous response | variance across scenarios | Helped |
| Use DualIV among kernel methods | Continuous NPIV regression · Continuous response | variance across scenarios | Hurt |
| Use Kernel SAGD-IV instead of Deep SAGD-IV | Binary NPIV regression · Binary response: linear and sine | log MSE | Helped |
| Use Deep SAGD-IV instead of Kernel SAGD-IV | Binary NPIV regression · Binary response | log MSE | Hurt |
Surprising findings
- Deep SAGD-IV had more difficulty uncovering the structural function in the binary-response setting than the kernel variant.
Limitations
- Consistency for the structural function is not established; it requires additional assumptions beyond excess-risk bounds.
- Kernel mean embedding estimator of the conditional expectation operator may not scale to big data.
- Binary-outcome case requires a known error distribution (logistic) and independence of the error from Z.
Experimental setup
Datasets: synthetic NPIV data generating process (continuous and binary response)
Benchmarks: synthetic NPIV DGP with step, abs, linear, sin response functions synthetic binary-response NPIV DGP (sin and linear)
Metrics: log mean squared error variance across realizations
Baselines: KIV DualIV DeepGMM DeepIV 2SLS
Builds on: KIV DeepIV DeepGMM DualIV
Closest papers by meaning
- Stochastic Optimization Algorithms for Instrumental Variable Regression with Streaming Data (NeurIPS 2024 · similarity 0.89)
- Demystifying Spectral Feature Learning for Instrumental Variable Regression (NeurIPS 2025 · similarity 0.87)
- Conditional Instrumental Variable Regression with Representation Learning for Causal Inference (ICLR 2024 · similarity 0.87)
- Data-Driven Selection of Instrumental Variables for Additive Nonlinear, Constant Effects Models (ICML 2025 · similarity 0.86)
- Stochastic Online Instrumental Variable Regression: Regrets for Endogeneity and Bandit Feedback (AAAI 2025 · similarity 0.86)
- Geometry-Aware Instrumental Variable Regression (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.