Research map / Adversarial robustness and security / Certified and empirical robustness
Geometry-Aware Instrumental Variable Regression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Sinkhorn Method of Moments, an optimal-transport empirical-likelihood IV estimator whose dual uses data gradients and Laplacians to be geometry-aware, improving robustness to corrupted data and adversarial attacks while staying competitive on standard IV benchmarks.
Problem
Instrumental variable regression via conditional moment restrictions relies on reweighting or sampling the empirical data, which is blind to data-space geometry and thus vulnerable to corrupted data and adversarial perturbations.
Approach
Replace the phi-divergence (reweighting) or MMD (sampling) in empirical likelihood with an entropy-regularized Sinkhorn optimal transport distance; a small-epsilon Taylor expansion of the dual yields an objective with data gradients and Laplacians of the moment violation, so the estimator favors instruments that are locally stable around data points.
Key results
- On NetworkIV, SMM matches SMD and VMM (e.g., sin: 0.15±0.01 vs SMD 0.12±0.01; step 0.04±0.00 vs SMD 0.04±0.01), while MMR fails on step (>100).
- In the corrupted-treatment experiment, SMM shows a small advantage with no corruption and scales more favorably than baselines as corruption grows.
- In the FGSM adversarial experiment, SMM is more robust than SMD and VMM, though MMR is the most robust, which the authors attribute to strong regularization.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Kernel-SMM instead of SMD | Kernel IV regression · NetworkIV sin | prediction error: 0.12\pm0.01 -> 0.15\pm0.01 | Hurt |
| Use Kernel-SMM instead of VMM | Kernel IV regression · NetworkIV abs | prediction error: 0.09\pm0.04 -> 0.12\pm0.04 | Hurt |
| Use Kernel-SMM instead of SMD | Kernel IV regression · NetworkIV step | prediction error: 0.04\pm0.01 -> 0.04\pm0.00 | No clear effect |
| Use Kernel-SMM instead of VMM | Kernel IV regression · NetworkIV linear | prediction error: 0.03\pm0.01 -> 0.07\pm0.03 | Hurt |
| Use Neural-SMM instead of DeepGMM | Neural IV regression · NetworkIV sin | prediction error: 0.08\pm0.01 -> 0.07\pm0.01 | Helped |
| Use Neural-SMM instead of DeepGMM | Neural IV regression · NetworkIV abs | prediction error: 0.04\pm0.01 -> 0.04\pm0.01 | No clear effect |
| Use Neural-SMM instead of DeepGMM | Neural IV regression · NetworkIV step | prediction error: 0.07\pm0.01 -> 0.07\pm0.01 | No clear effect |
| Use Neural-SMM instead of DeepGMM | Neural IV regression · NetworkIV linear | prediction error: 0.05\pm0.01 -> 0.05\pm0.01 | No clear effect |
| Use SMM instead of related estimators as the corrupted-treatment proportion increases | Kernel IV regression · SimpleIV with corrupted treatment data | mean-squared error | Helped |
| Use SMM instead of SMD or VMM as FGSM attack strength increases | Neural IV regression · Adversarially robust IV regression | mean-squared error | Helped |
Surprising findings
- MMR, which was worst in the corrupted-data experiments, shows the least sensitivity to adversarial perturbations.
- MMR completely fails on the step function in NetworkIV (MSE above 100).
Limitations
- Kernel version scales poorly to large sample sizes
- Neural-SMM is only partially explored and deferred to appendix; its detailed analysis is left for future work
- Consistency relies on compactness and smoothness assumptions that may be violated in practice
Experimental setup
Datasets: SimpleIV synthetic process (Kremer et al.) NetworkIV synthetic benchmark
Benchmarks: SimpleIV with corrupted treatments FGSM adversarial IV experiment NetworkIV
Metrics: mean squared error of prediction f(T) vs f(T; theta_0)
Baselines: MMR SMD VMM LSQ DeepGMM FunctionalGEL
Builds on: generalized empirical likelihood Sinkhorn distributionally robust optimization two-stage GMM variational method of moments
Closest papers by meaning
- Nonparametric Instrumental Variable Regression through Stochastic Approximate Gradients (NeurIPS 2024 · similarity 0.86)
- Stochastic Optimization Algorithms for Instrumental Variable Regression with Streaming Data (NeurIPS 2024 · similarity 0.85)
- Demystifying Spectral Feature Learning for Instrumental Variable Regression (NeurIPS 2025 · similarity 0.84)
- Optimality and Adaptivity of Deep Neural Features for Instrumental Variable Regression (ICLR 2025 · similarity 0.84)
- Exploiting Geometry for Treatment Effect Estimation via Optimal Transport (AAAI 2024 · similarity 0.83)
- Robust Sparse Regression with Non-Isotropic Designs (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.