Research map / Reinforcement learning / Imitation learning
When to Act and When to Ask: Policy Learning With Deferral Under Hidden Confounding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CARED, a causal action recommendation method that learns when to act or defer to a human expert from observational data under bounded hidden confounding (MSM), using CAPO bounds from B-learner to form cost-sensitive deferral losses.
Problem
Learning causal treatment policies with an option to defer to a human expert from observational data where hidden confounders may bias estimates; labels for the optimal action are unobserved.
Approach
Extends the Mozannar and Sontag consistent cost-sensitive deferral surrogate to causal settings by replacing unobserved true labels with costs built from sharp CAPO bounds under the Marginal Sensitivity Model, with an expert-deferral cost derived from observed expert actions and outcomes.
Key results
- On synthetic data, CARED improves over the human expert for all sensitivity values Λ, while ConfHAI and CRLogit beat the expert only in a narrow range around the true Λ0.
- On IHDP hidden confounding, CARED consistently outperforms the expert policy across Λ and achieves policy value closest to the Oracle policy.
- At equivalent deferral rates on IHDP, CARED outperforms ConfHAI, B-learner policy and random deferral, peaking near a deferral rate of about 0.6.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CARED instead of the expert policy | CARED · Synthetic data | policy regret | Helped |
| Use ConfHAI or CRLogit instead of the expert policy | ConfHAI and CRLogit · Synthetic data | policy regret | Mixed |
| Use CARED instead of the expert policy | CARED · IHDP Hidden Confounding | policy value | Helped |
| Use CARED instead of other methods at equivalent deferral rates | CARED · IHDP Hidden Confounding | policy value | Helped |
| Increase the assumed confounding level Λ | CARED · IHDP Hidden Confounding | policy value | Mixed |
Surprising findings
- ConfHAI defers only a small fraction of samples on average, suggesting it does not learn the expert's strengths and weaknesses.
Limitations
- Requires access to expert actions in observational data and cannot accommodate a different expert whose potential outcomes are unknown.
- Assumes experts behave the same on deferred cases at test time as before deployment, ignoring behavior change from deployment.
- Choice of Λ is non-trivial; performance trends toward the expert policy at high Λ due to wide CAPO intervals.
Experimental setup
Datasets: Synthetic data from Gao and Yin 2023 IHDP hidden confounding (Jesson et al. 2021)
Benchmarks: Synthetic data (Gao and Yin 2023 DGP) IHDP hidden confounding
Metrics: policy regret policy value deferral rate
Baselines: ConfHAI (Gao and Yin 2023) CRLogit (Kallus and Zhou 2020) Pessimistic Policy B-Learner Policy Random Deferral Policy Oracle Policy Expert Policy
Builds on: Mozannar and Sontag 2020 consistent surrogate for learning to defer B-learner Marginal Sensitivity Model Athey and Wager 2021 policy learning costs Gao and Yin 2023 ConfHAI
Closest papers by meaning
- Confounding-Robust Deferral Policy Learning (AAAI 2025 · similarity 0.89)
- Towards Safe Policy Learning under Partial Identifiability: A Causal Approach (AAAI 2024 · similarity 0.86)
- Regression with Multi-Expert Deferral (ICML 2024 · similarity 0.84)
- Behavior Cloning is Not All You Need: The Optimality of On-Policy Distillation for Noisy Expert Feedback (NeurIPS 2026 · similarity 0.84)
- SafeAR: Safe Algorithmic Recourse by Risk-Aware Policies (AAAI 2024 · similarity 0.84)
- Principled Bayesian Optimization in Collaboration with Human Experts (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.