Research map / Probabilistic and scientific modeling / Treatment effects
Demystifying Spectral Feature Learning for Instrumental Variable Regression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives a generalization bound for sieve 2SLS with spectral features for nonparametric IV, proves the contrastive spectral loss of prior work equals a Hilbert-Schmidt low-rank approximation of the conditional expectation operator, and gives a good/bad/ugly taxonomy validated on synthetic and dSprites experiments.
Problem
Causal effect estimation with hidden confounders via nonparametric instrumental variable regression, where learned spectral features are used in two-stage least squares; the paper asks when and why this works and when it fails.
Approach
Shows that spectral features minimize the sieve measure of ill-posedness, derives a generalization bound decomposed into spectral alignment (approximation error) and ill-posedness (singular value decay), proves the spectral contrastive loss is equivalent to a Hilbert-Schmidt rank-d approximation of the conditional expectation operator, and defines a good/bad/ugly taxonomy.
Key results
- Generalization bound for sieve 2SLS with spectral features: error O_p(approximation error + sqrt(d/(n sigma_d^2))), with rate n^{-r/(2(r+1)+1/p)} under polynomial decay and source conditions.
- Synthetic experiments show MSE decreases with slower singular value decay and stronger spectral alignment, for both oracle and learned features.
- Spectral contrastive loss is shown equal to a Hilbert-Schmidt approximation objective, so its minimizer spans the top-d singular subspaces (Eckart-Young-Mirsky).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use slower singular-value decay | Sieve 2SLS with oracle spectral features · Synthetic NPIV | MSE | Helped |
| Improve alignment of the structural function with the leading singular functions | Sieve 2SLS with oracle spectral features · Synthetic NPIV | MSE | Helped |
| Use slower singular-value decay | Sieve 2SLS with learned spectral features · Synthetic NPIV | MSE | Helped |
| Improve alignment of the structural function with the leading singular functions | Sieve 2SLS with learned spectral features · Synthetic NPIV | MSE | Helped |
| Use learned rather than oracle spectral features | Sieve 2SLS · Synthetic NPIV | MSE | Hurt |
| Concentrate the structural function on the top singular functions when singular-value decay is fast | Sieve 2SLS with learned spectral features · Synthetic NPIV | MSE | Helped |
| Place the structural function orthogonal to the singular functions of the operator | Spectral methods · Synthetic NPIV, misaligned regime | recovery | Hurt |
Limitations
- Learned-feature error term epsilon_d is not controlled in terms of sample size or architecture; left to future work.
- Rank-d factorization of the density is only an approximation in practice; exact factorization holds only if rank(T) <= d.
- Theory assumes linear independence of learned features and a link condition; the dSprites diagnosis relies on estimated alignment.
Experimental setup
Datasets: dSprites synthetic NPIV data
Benchmarks: synthetic NPIV settings dSprites
Metrics: mean squared error
Baselines: end-to-end 2SLS feature learning saddle-point IV estimator fixed-basis sieve 2SLS
Builds on: spectral contrastive learning sieve 2SLS Eckart-Young-Mirsky theorem
Closest papers by meaning
- Conditional Instrumental Variable Regression with Representation Learning for Causal Inference (ICLR 2024 · similarity 0.87)
- Nonparametric Instrumental Variable Regression through Stochastic Approximate Gradients (NeurIPS 2024 · similarity 0.87)
- Stochastic Optimization Algorithms for Instrumental Variable Regression with Streaming Data (NeurIPS 2024 · similarity 0.86)
- Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regression (ICLR 2025 · similarity 0.85)
- An Analysis of Causal Effect Estimation using Outcome Invariant Data Augmentation (NeurIPS 2025 · similarity 0.85)
- Geometry-Aware Instrumental Variable Regression (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.