Research map / Reinforcement learning / Visual and representation RL
Contextualized Policy Recovery: Modeling and Interpreting Medical Decisions with Adaptive Imitation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Contextualized Policy Recovery (CPR), which uses a black-box recurrent history encoder to generate context-specific linear (logistic) observation-to-action policies, achieving near black-box accuracy with interpretable models on MIMIC antibiotics and ADNI MRI prescription.
Problem
Interpretable policy learning from demonstrated clinical behavior forces a tradeoff between accuracy and interpretability, since a single universal policy must be either complex or too simple to capture heterogeneous human decisions.
Approach
Treat each context (patient history and covariates) as its own task: a recurrent encoder generates the parameters of a simple logistic policy at each timestep, so the complex overall policy is a piecewise combination of interpretable context-specific linear models; a global variant telescopes these into a single linear model with exact per-feature effects.
Key results
- On ADNI MRI scans, CPR-RNN reaches 0.72 AUROC versus 0.66 for logistic regression and 0.62 for POETREE, matching black-box RNN (0.72).
- On MIMIC antibiotics (7 features), CPR-RNN and CPR-LSTM reach 0.82 AUROC, versus 0.57 for logistic regression and 0.68 for POETREE, close to black-box RNN/LSTM (0.83/0.84).
- Static contexts (age, gender) raise CPR AUROC on ADNI to 0.763.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CPR-RNN instead of logistic regression | ADNI MRI policy prediction · ADNI MRI scans | AUROC: 0.66 ± 0.03 -> 0.72 ± 0.02 | Helped |
| Use CPR-RNN instead of logistic regression | ADNI MRI policy prediction · ADNI MRI scans | AUPRC: 0.86 ± 0.01 -> 0.88 ± 0.02 | Helped |
| Use CPR-RNN instead of LSTM | ADNI MRI policy prediction · ADNI MRI scans | AUROC: 0.71 ± 0.02 -> 0.72 ± 0.02 | No clear effect |
| Use CPR-RNN instead of logistic regression | MIMIC antibiotic policy prediction · MIMIC Antibiotics | AUROC: 0.57 ± 0.03 -> 0.82 ± 0.01 | Helped |
| Use CPR-RNN instead of logistic regression | MIMIC antibiotic policy prediction · MIMIC Antibiotics | AUPRC: 0.80 ± 0.03 -> 0.90 ± 0.01 | Helped |
| Use CPR-RNN instead of LSTM | MIMIC antibiotic policy prediction · MIMIC Antibiotics | AUROC: 0.84 ± 0.01 -> 0.82 ± 0.01 | Hurt |
| Add static contexts (age and gender) to the CPR encoder | CPR on ADNI MRI policy prediction · ADNI MRI scans | AUROC: 0.763 | Helped |
Surprising findings
- A condition-specific logistic regression baseline performs nearly as well as CPR and black-box models on the ADNI canonical task (AUROC 0.71), suggesting limited heterogeneity in that task's context features.
Limitations
- Evaluated only in offline, partially observable settings; online RL extension is claimed but not tested.
- Simple context-specific linear policies depend on the chosen encoder and hyperparameters; ADNI discrete contexts limit heterogeneity analysis.
- Baseline numbers for INTERPOLE and POETREE are taken from prior reports rather than re-run, and some results use different feature sets.
Experimental setup
Datasets: MIMIC-III ADNI
Benchmarks: MIMIC-III antibiotics prescription ADNI MRI prescription
Metrics: AUROC AUPRC Brier score
Baselines: Logistic regression INTERPOLE POETREE RNN LSTM condition-specific logistic regression
Builds on: POETREE INTERPOLE contextualized explanation networks varying coefficient models
Closest papers by meaning
- Explaining by Imitating: Understanding Decisions by Interpretable Policy Learning (ICLR 2021 · similarity 0.89)
- Towards Safe Policy Learning under Partial Identifiability: A Causal Approach (AAAI 2024 · similarity 0.84)
- EpiCare: A Reinforcement Learning Benchmark for Dynamic Treatment Regimes (NeurIPS 2024 · similarity 0.84)
- In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought (ICML 2024 · similarity 0.84)
- Position: Reinforcement Learning in Dynamic Treatment Regimes Needs Critical Reexamination (ICML 2024 · similarity 0.84)
- When to Act and When to Ask: Policy Learning With Deferral Under Hidden Confounding (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.