Research map / Reinforcement learning / Policy gradient and actor-critic
Optimizing Vital Sign Monitoring in Resource-Constrained Maternal Care: An RL-Based Restless Bandit Approach
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Frames allocation of scarce wireless vital-sign monitors to postpartum mothers as a constrained restless bandit and learns an allocation policy with PPO actor-critic trained on a simulator built from MIMIC-III/IV, beating heuristics by up to 4x.
Problem
Allocating scarce wireless vital sign monitoring devices to mothers after childbirth under minimum/maximum monitoring durations, no reassignment, and no patient features, with unknown continuous transition dynamics.
Approach
Models device allocation as a restless multi-armed bandit with novel allocation constraints (min/max duration, no re-assignment, arrivals/departures) and learns a shared actor-critic policy with PPO over continuous vital-sign states, trained on a simulator fit to patient data.
Key results
- On MIMIC-IV with B=3, N=20 the method outperforms the second-best baseline by 173%; on MIMIC-III (B=3, N=20) by 433%.
- On Mbarara data, the method significantly beats Random and Highest Variability, but the gap with Extreme Values is not statistically significant (29% higher for B=3, N=20).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PPO-based allocation instead of the second-best heuristic baseline | Device allocation policy · MIMIC-III (B=3, N=20) | normalized reward: 433% | Helped |
| use PPO-based allocation instead of the second-best heuristic baseline | Device allocation policy · MIMIC-IV (B=3, N=20) | normalized reward: 173% | Helped |
| use PPO-based allocation instead of the second-best heuristic baseline | Device allocation policy · MIMIC-III (B=6, N=50) | normalized reward: 431% | Helped |
| use PPO-based allocation instead of the second-best heuristic baseline | Device allocation policy · MIMIC-IV (B=6, N=50) | normalized reward: 141% | Helped |
| use PPO-based allocation instead of Extreme Values | Device allocation policy · Mbarara maternal-care simulations (B=3, N=20; B=7, N=40) | normalized reward: 29%; 20% | No clear effect |
| use PPO-based allocation instead of Random | Device allocation policy · Mbarara maternal-care simulations | normalized reward | Helped |
| use PPO-based allocation instead of Highest Variability | Device allocation policy · Mbarara maternal-care simulations | normalized reward | Helped |
Surprising findings
- Intuitive heuristics Extreme Values and Highest Variability perform worse than Random on the MIMIC simulations.
- On the small Mbarara dataset the random strategy is competitive and the advantage over Extreme Values is not statistically significant.
Limitations
- All experiments are in simulation; the simulator is fit on only 100 patient traces for Mbarara.
- Simulator assumes Gaussian transitions and intervention effects with hand-set parameters; no real-world deployment.
- Only one-sided reward thresholds are used due to monotonic reward function constraint.
Experimental setup
Datasets: MIMIC-III MIMIC-IV Mbarara Regional Referral Hospital vital sign trajectories (100 patients)
Benchmarks: MIMIC-III simulation MIMIC-IV simulation Mbarara simulator
Metrics: normalized discounted cumulative reward (minus No Action, di
Baselines: No Action Random Extreme Values Highest Variability
Builds on: Proximal Policy Optimization restless multi-armed bandits streaming restless bandits
Closest papers by meaning
- A Decision-Language Model (DLM) for Dynamic Restless Multi-Armed Bandit Tasks in Public Health (NeurIPS 2024 · similarity 0.86)
- Improving Health Information Access in the World’s Largest Maternal Mobile Health Program via Bandit Algorithms (AAAI 2024 · similarity 0.86)
- Context in Public Health for Underserved Communities: A Bayesian Approach to Online Restless Bandits (AAAI 2025 · similarity 0.84)
- RoME: A Robust Mixed-Effects Bandit Algorithm for Optimizing Mobile Health Interventions (NeurIPS 2024 · similarity 0.83)
- PRIORITY2REWARD: Incorporating Healthworker Preferences for Resource Allocation Planning (AAAI 2025 · similarity 0.81)
- Global Rewards in Restless Multi-Armed Bandits (NeurIPS 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.