Research map / Reinforcement learning / MDPs and planning
Semi-Markovian Planning to Coordinate Aerial and Maritime Medical Evacuation Platforms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Models aerial-maritime medical evacuation with watercraft exchange points as a semi-Markov decision process solved by MCTS with root parallelization, validated in simulation and a real HH-60M/LSV mock patient transfer in Hawaii.
Problem
Coordinating multiple aerial and underway maritime platforms to select patient exchange points and dispatch medical evacuation aircraft so as to minimize incident response time across islands with uneven casualty demand.
Approach
Treats moving watercraft and fixed land locations as exchange points in a semi-Markov decision process with a fused Weibull-survival and linear clearance reward, and solves it online with MCTS using root parallelization over simulated casualty request threads.
Key results
- An optimal policy with watercraft exchange points outperforms an optimal policy without them and a greedy policy by 35% and 40% respectively.
- Watercraft exchange points cut average incident response times by 50% for two UH-1 aircraft at a 30% casualty increase, versus 14-16% for SB-1 Defiant and V-280 Valor.
- The real-world deployment incident response time from request to role three delivery was 76 minutes, with 16 minutes caused by maritime traffic delays.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add moving watercraft exchange points to the optimal policy | Optimal evacuation policy · Simulation scenarios | total reward: 30-35% | Helped |
| Use an optimal policy with watercraft exchange points instead of an optimal policy without them | Evacuation planning · Simulation scenarios | performance: 35% | Helped |
| Use an optimal policy with watercraft exchange points instead of a greedy policy | Evacuation planning · Simulation scenarios | performance: 40% | Helped |
| Use an optimal policy without watercraft exchange points instead of a greedy policy | Evacuation planning · Simulation scenarios | total reward: 10-15% | Helped |
| Increase casualty magnitude above 1.2 | All three considered policies · Simulation scenarios | total reward: above a magnitude multiplier of 1.2 | Hurt |
| Increase the forward-island casualty proportion multiplier above 1.1 and make watercraft exchange points available | Optimal evacuation policy · Simulation scenarios | total reward: above 1.1 | Helped |
| Use watercraft exchange points when Kauai casualties exceed Oahu casualties by 25% or more | Optimal evacuation policy · Simulation scenarios with more casualties on Kauai | performance: 25% or more | Helped |
| Increase patients per evacuation request to as many as four | Optimal policies with and without watercraft exchange points · Simulation scenarios | total reward: up to four patients | Helped |
| Increase patients per evacuation request to five or more | Optimal policy · Simulation scenarios | total reward: five or more patients per request | Hurt |
| Increase the proportion of interisland transfer patients | Optimal policies with and without watercraft exchange points · Simulation scenarios | total reward | No clear effect |
| Make watercraft exchange points available | Evacuation planning with aircraft at different cruise airspeeds · Simulation scenarios | average incident response time: 50% | Helped |
| Make watercraft exchange points available | SB-1 Defiant evacuation aircraft · Simulation scenarios with a 30% increase in casualties | incident response time: 16% | Helped |
Surprising findings
- Faster aircraft reduce the benefit of watercraft exchange points on response times.
- Watercraft exchange points nearly eliminate response time disparity between platoons without any control over watercraft location.
Limitations
- Assumes full observability, requiring platforms to share state and intentions reliably.
- Deployment was a single mock patient transfer with a manikin rather than an actual patient evacuation.
- Simulation relies on a generated casualty model and assumed parameter ranges.
Experimental setup
Datasets: simulated casualty generation data (Poisson patient arrivals)
Benchmarks: simulated Hawaiian Islands evacuation scenarios HH-60M and LSV mock patient transfer deployment
Metrics: total reward incident response time ratio of watercraft transfers
Baselines: optimal policy without watercraft exchange points greedy dispatch policy
Builds on: Monte Carlo tree search UCT root parallelization semi-Markov decision process
Closest papers by meaning
- Multi-Agent Reinforcement Learning with Hierarchical Coordination for Emergency Responder Stationing (ICML 2024 · similarity 0.82)
- Deterministic versus Probabilistic Methods for Searching for an Evasive Target (AAAI 2017 · similarity 0.81)
- AoI-MDP: An AoI Optimized Markov Decision Process Dedicated in the Underwater Task (Student Abstract) (AAAI 2025 · similarity 0.80)
- A Distributional Framework for Risk-Sensitive End-to-End Planning in Continuous MDPs (AAAI 2022 · similarity 0.79)
- Sequential Monte Carlo for Policy Optimization in Continuous POMDPs (NeurIPS 2025 · similarity 0.78)
- Augmented Bayesian Policy Search (ICLR 2024 · similarity 0.78)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.