Research map / Probabilistic and scientific modeling / Treatment effects
Estimating Distributional Treatment Effects in Randomized Experiments: Machine Learning for Variance Reduction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a regression-adjusted (Neyman-orthogonal, cross-fitted) estimator for distributional treatment effects (DTE, PTE, QTE) in randomized experiments, using ML nuisance estimates to reduce variance, with uniform inference via multiplier bootstrap.
Problem
Estimating distributional treatment effects (DTE, PTE, QTE) in randomized experiments precisely, given pre-treatment covariates that may be high-dimensional, without relying only on average effects.
Approach
Adjust the empirical distribution function with cross-fitted ML estimates of conditional outcome distributions given covariates (distribution regression), framed as a Neyman-orthogonal moment condition; this provably weakly reduces variance versus the simple estimator and yields uniformly valid inference via a multiplier bootstrap.
Key results
- In simulations with n=5000, the ML-adjusted DTE estimator reduces RMSE by roughly 40-50% relative to the simple estimator, versus about 15-35% for linear adjustment.
- Bias of all DTE estimators is at most about 2% of the true value even at n=500.
- In the water-nudge application, the regression-adjusted DTE has substantially tighter confidence intervals than the simple DTE, especially between 15 and 110 (thousand gallons), and the PTE shows a significant decrease in the 70-80 range that is insignificant under simple estimates.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use logistic LASSO regression adjustment instead of the simple empirical distribution estimator | DTE estimation · Synthetic simulation, n=5000 | RMSE: around 40-50% RMSE reduction | Helped |
| Use linear regression adjustment instead of the simple empirical distribution estimator | DTE estimation · Synthetic simulation, n=5000 | RMSE: roughly 15-35% RMSE reduction | Helped |
| Use logistic LASSO rather than linear regression for adjustment | DTE estimation · Synthetic simulation | RMSE | Helped |
| Use regression adjustment rather than the simple empirical distribution estimator | DTE estimation · Synthetic simulation | RMSE | Helped |
| Use gradient-boosting regression adjustment rather than the simple estimator | DTE estimation · Water-consumption experiment | variance | Helped |
| Use gradient-boosting regression adjustment rather than the simple estimator | PTE estimation · Water-consumption experiment | statistical significance of the probability change for (70, 80] | Helped |
| Use the ideal regression-adjusted estimator with the true conditional distribution instead of the simple empirical distribution estimator | Outcome distribution estimation · Randomized experiments | variance | Helped |
Surprising findings
- The regression-adjusted PTE shows a significant decrease in the (70,80] range that is not significant under the simple estimator.
Limitations
- Assumes perfect compliance and no interference (no network or peer effects)
- Variance reduction depends on pre-treatment covariates being highly predictive of the outcome
- Only applies to experimental data already collected; no design-stage variance reduction
Experimental setup
Datasets: simulated i.i.d. data Cobb County water-nudge randomized experiment (2007), n=78,500
Benchmarks: simulation study (n=500, 1000, 5000) Cobb County water consumption experiment
Metrics: bias as % of true value RMSE reduction (%) pointwise and uniform confidence bands
Baselines: simple empirical distribution estimator linear regression adjustment
Builds on: Neyman-orthogonal moment conditions (double/debiased ML) cross-fitting regression adjustment for ATE (Lin 2013 style) functional delta method multiplier bootstrap
Closest papers by meaning
- On Efficient Estimation of Distributional Treatment Effects under Covariate-Adaptive Randomization (ICML 2025 · similarity 0.93)
- Beyond the Average: Distributional Causal Inference under Imperfect Compliance (NeurIPS 2025 · similarity 0.89)
- Efficient Randomized Experiments Using Foundation Models (NeurIPS 2025 · similarity 0.86)
- Active Adaptive Experimental Design for Treatment Effect Estimation with Covariate Choice (ICML 2024 · similarity 0.85)
- Estimating Heterogeneous Treatment Effects by Combining Weak Instruments and Observational Data (NeurIPS 2024 · similarity 0.85)
- Benchmarking Estimators for Natural Experiments: A Novel Dataset and a Doubly Robust Algorithm (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.