Research map / Reinforcement learning / Offline RL
Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ConformalDQN, a Double DQN extended with a behavior-policy probability network and a conformal predictor that restricts action selection to confident actions, for offline mechanical ventilation control on MIMIC-IV.
Problem
Offline RL for ICU mechanical ventilation suffers from distribution shift, Q-value overestimation and out-of-distribution actions, which can lead to unsafe ventilator settings.
Approach
Combine a Double DQN with a learned behavior-policy action-probability network and a split-conformal threshold, so the agent picks the highest-Q action among actions the behavior policy would plausibly take, falling back to argmax Q when none qualify; adds an NLL term and L2 logit regularization.
Key results
- ConformalDQN achieved mean initial Q-value 0.639 ± 0.026 versus 0.581 ± 0.025 for DeepVent (CQL) and 0.491 ± 0.001 for physicians, with estimated survival 83.89% vs 81.65% vs 74.90%.
- Q-value to mortality correlation was -0.563 for ConformalDQN vs -0.429 for CQL and -0.410 for DQN.
- Standard DDQN overestimated Q-values especially on OOD data, while CQL and ConformalDQN stayed below the 1.0 threshold.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add conformal prediction and uncertainty-aware action selection to DDQN | DDQN for mechanical ventilation · MIMIC-IV test set | mean initial state value: 0.581 $\pm$ 0.025 -> 0.639 $\pm$ 0.026 | Helped |
| Use ConformalDQN instead of the physician policy | Mechanical ventilation policy · MIMIC-IV test set | estimated 90-day survival rate: 74.90 -> 83.89 | Helped |
| Add conformal prediction and uncertainty-aware action selection to DDQN | DDQN for mechanical ventilation · MIMIC-IV test set | Q-value–mortality correlation: -0.410 $\pm$ 0.011 -> -0.563 $\pm$ 0.003 | Mixed |
Surprising findings
- Survival-rate mapping is derived from FQE Q-values binned against physician outcomes, so absolute survival estimates depend on that calibration rather than observed outcomes.
Limitations
- Discretized action space of 343 actions; continuous actions not addressed.
- Evaluation relies on FQE estimates rather than prospective or real patient outcomes.
- Conformal threshold is global rather than state-conditioned.
Experimental setup
Datasets: MIMIC-IV
Benchmarks: MIMIC-IV held-out test set (in-distribution and out-of-distribution)
Metrics: Fitted Q-Evaluation estimated 90-day survival (mean initial Pearson correlation between Q-values and mortality action distribution (PEEP, FiO2, tidal volume) overestimation relative to threshold of 1.0
Baselines: Physician policy DeepVent (CQL) DDQN Behavior cloning DQN
Builds on: Double DQN Conservative Q-Learning DeepVent Fitted Q-Evaluation split conformal prediction
Closest papers by meaning
- Offline Guarded Safe Reinforcement Learning for Medical Treatment Optimization Strategies (NeurIPS 2025 · similarity 0.83)
- Yet Another ICU Benchmark: A Flexible Multi-Center Framework for Clinical ML (ICLR 2024 · similarity 0.83)
- Reliable Uncertainty Quantification in Machine Learning via Conformal Prediction (AAAI 2025 · similarity 0.83)
- Multi-Source Conformal Inference Under Distribution Shift (ICML 2024 · similarity 0.83)
- Robust Uncertainty Quantification Using Conformalised Monte Carlo Prediction (AAAI 2024 · similarity 0.82)
- Uncertainty Quantification for Deep Regression using Contextualised Normalizing Flows (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.