Research map / Reinforcement learning / Policy gradient and actor-critic
Distributionally Robust Policy Evaluation and Learning for Continuous Treatment with Observational Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a kernel-based IPW estimator for distributionally robust (KL-ball) policy evaluation and learning under continuous treatments, with asymptotic and finite-sample regret analysis plus simulation and Warfarin experiments.
Problem
Offline policy evaluation and learning with continuous treatments when the deployment distribution differs from the data-generating distribution (distribution shift); existing IPW methods target discrete treatments.
Approach
Replace the Dirac delta in the IPW form of the KL-robust (exponential-tilting dual) value with a scaled kernel K_h of bandwidth h, yielding a tractable, normalized kernel IPW estimator of the distributionally robust value and policy, with asymptotic normality and regret bounds.
Key results
- In simulation, the continuous-treatment robust policy achieves mean robust value 6.24 at eta_test=0.05 versus 5.83-5.88 for discretized (2-4 bin) policies, close to optimal 6.41.
- Robust policy (eta=0.2) has lower value sensitivity to perturbation than the non-robust policy across N_train from 500 to 2500, with the robust policy improving with more samples.
- On Warfarin, robust policies with eta in 0.4-0.5 give mean value about 6.41-6.45 vs 6.377 for the non-robust policy, and most robust policies beat the non-robust one in lower percentiles.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use continuous-treatment DRO instead of discretizing treatment into bins | DRO policy learning · Simulation | robust value: 5.88±0.15 -> 6.24±0.32 | Helped |
| Train a distributionally robust policy rather than a nonrobust policy | Linear policy class · Simulation under data perturbation | perturbed reward: 5.02±10.37 -> 5.48±15.46 | Helped |
| Increase training sample size for the robust policy | Robust policy, η=0.2 · Simulation, perturbed reward | perturbed reward: 4.94±15.64 -> 5.46±15.46 | Helped |
| Increase ambiguity radius from η=0.3 to η=0.4 or η=0.5 | Warfarin DRO policy · Warfarin age-shift test set | mean reward: 6.372 -> 6.454 | Mixed |
Surprising findings
- The robust policy with eta=0.3 yields a mean comparable to the non-robust policy, and the expected reward is non-monotone in eta, peaking at intermediate values and declining for larger eta.
Limitations
- Relies on KL divergence; the choice of ambiguity radius eta is left open.
- Policy class restricted to linear policies, which may miss complex relationships.
- Uses plain IPW without doubly robust correction, so convergence rate may be slower.
Experimental setup
Datasets: simulated data (uniform covariates, continuous treatment) Warfarin dataset (5528 patients, semi-synthetic outcomes)
Benchmarks: simulated continuous-treatment DGP Warfarin semi-synthetic study
Metrics: distributionally robust value estimate perturbation-based robust value sample mean potential outcome percentiles of policy value
Baselines: non-robust IPW policy learning discretized-treatment DRO policies (k=2,3,4 bins)
Builds on: inverse probability weighting for discrete-treatment policy learning KL-divergence distributionally robust optimization dual kernel-based continuous-treatment IPW
Closest papers by meaning
- Distributionally Robust Policy Learning under Concept Drifts (ICML 2025 · similarity 0.87)
- Off-policy estimation with adaptively collected data: the power of online learning (NeurIPS 2024 · similarity 0.86)
- Towards Safe Policy Learning under Partial Identifiability: A Causal Approach (AAAI 2024 · similarity 0.85)
- Assessing the robustness of heterogeneous treatment effects in survival analysis under informative censoring (NeurIPS 2026 · similarity 0.84)
- Doubly-Robust Estimation of Counterfactual Policy Mean Embeddings (NeurIPS 2025 · similarity 0.84)
- Distributionally Robust Reinforcement Learning with Interactive Data Collection: Fundamental Hardness and Near-Optimal Algorithms (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.