Research map / Probabilistic and scientific modeling / Treatment effects
Adaptive Instrument Design for Indirect Experiments
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DIA, an adaptive instrument-allocation method for indirect experiments that uses influence functions for leave-one-out MSE control variates and multi-rejection sampling to optimize a data collection policy over instruments for estimating non-linear IV/CATE estimators.
Problem
Sample-efficient estimation of treatment effects in indirect experiments, where treatment is only encouraged via instrumental variables, and instrument allocation is chosen without adaptive design, especially with many instruments and non-linear estimators.
Approach
Gradient-based search over a data-collection policy over instruments, with variance reduction via influence-function estimates of leave-one-out MSE (control variates computed without retraining) and multi-rejection sampling to reuse historical data under a new policy without exponential importance-weight variance.
Key results
- DIA improves the base estimator's accuracy by a factor of 1.2x to 10x over uniform allocation across domains.
- For some target accuracies, DIA needs only 80% to an order-of-magnitude less sample data than uniform.
- Gains show a U-shaped trend with number of instruments: largest potential gains with many instruments but harder policy learning when instruments are very numerous.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| adaptively allocate instruments with DIA instead of uniform allocation | 2SLS indirect experiment design · IV | MSE: 1.2× to 10× | Helped |
| adaptively allocate instruments with DIA instead of uniform allocation | Logistic regression conditional IV estimation · Conditional IV | MSE: 1.2× to 10× | Helped |
| adaptively allocate instruments with DIA instead of uniform allocation | Neural network IV estimation · TripAdvisor | MSE: 1.2× to 10× | Helped |
| use an RCT adaptive-design strategy naively for indirect experiments instead of DIA | Indirect experiment design · Indirect experiments | MSE | Hurt |
| increase the number of policy re-allocations (ablation) | DIA · IV, Conditional IV, and TripAdvisor | MSE | Helped |
| vary the number of available instruments (ablation) | DIA · IV, Conditional IV, and TripAdvisor | MSE | Mixed |
| change the rejection-sampling dataset size k (ablation) | Multi-rejection sampling policy evaluation · Policy MSE ordering | MSE | No clear effect |
Surprising findings
- Naively applying RL-style adaptive experiment design from direct experiments can perform worse than uniform sampling in indirect experiments.
Limitations
- Experiments restricted to relatively small synthetic and semi-synthetic settings
- Assumes homogeneous treatment effects (conditional LATE equals CATE)
- Does not yet handle natural-language or high-dimensional instruments
Experimental setup
Datasets: TripAdvisor customer data (simulator)
Benchmarks: IV simulator conditional IV simulator TripAdvisor simulator
Metrics: relative MSE versus uniform allocation
Baselines: uniform instrument allocation oracle brute-force search (IV setting) naive REINFORCE gradient importance sampling
Builds on: DeepIV two-stage least squares REINFORCE influence functions
Closest papers by meaning
- Efficient Adaptive Experimentation with Noncompliance (NeurIPS 2025 · similarity 0.91)
- Adaptive Data-Borrowing for Improving Treatment Effect Estimation using External Controls (NeurIPS 2025 · similarity 0.89)
- Active Adaptive Experimental Design for Treatment Effect Estimation with Covariate Choice (ICML 2024 · similarity 0.89)
- Fast Proxy Experiment Design for Causal Effect Identification (NeurIPS 2024 · similarity 0.86)
- Targeted Sequential Indirect Experiment Design (NeurIPS 2024 · similarity 0.85)
- Data-Driven Selection of Instrumental Variables for Additive Nonlinear, Constant Effects Models (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.