Research map / Multi-agent systems and games / Cooperative multi-agent RL
Multi-Agent Reinforcement Learning with Hierarchical Coordination for Emergency Responder Stationing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Replaces the MCTS low-level and high-level planners of a hierarchical emergency responder repositioning framework with DDPG actor-critic agents (transformer-based low-level actors, combinatorial discretization), cutting per-decision time from minutes to about 0.22 s while slightly reducing response times on Nashville data.
Problem
Proactive repositioning of emergency responders (ambulances) among depots is combinatorially large and uncertain; the best existing hierarchical MCTS method needs minutes per decision, too slow for emergency response.
Approach
Keep the spatial hierarchical decomposition but replace online MCTS with DDPG-trained actor-critic agents: transformer-XL low-level actors handle variable-size responder sets, continuous actor outputs are mapped to discrete allocations via maximum-weight matching (low level) and min-cost flow (high level), and the high-level reward is estimated from low-level critic values.
Key results
- Per-decision computation time about 0.22 s versus about 3 minutes for the MCTS baseline, roughly three orders of magnitude faster.
- Average response time about 5 seconds lower than MCTS with 5-region segmentation and 13 seconds lower with 7-region segmentation.
- A centralized variant of the learned approach performs 66 seconds worse than the hierarchical approach; DRLSN response times are about 50% worse.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace online MCTS planning with hierarchical DDPG agents | Emergency responder stationing in Nashville · Nashville decision-making | decision time: 3 minutes -> 0.22 seconds | Helped |
| replace MCTS with hierarchical DDPG | Emergency responder stationing · Nashville 5-region segmentation | average response time: 5 seconds lower than MCTS | Helped |
| replace MCTS with hierarchical DDPG | Emergency responder stationing · Nashville 7-region segmentation | average response time: 13 seconds lower than MCTS | Helped |
| replace the TrXL actor with a GTrXL actor (ablation) | Low-level agent with the prior-work HLP · Nashville, 26 responders | average response time: 5 seconds higher than TrXL | Hurt |
| replace the TrXL actor with an LSTM actor (ablation) | Low-level agent with the prior-work HLP · Nashville, 26 responders | average response time: 18 seconds higher than TrXL | Hurt |
| use a single region instead of hierarchical coordination (ablation) | Hierarchical DDPG · Nashville | average response time: 66 seconds worse than hierarchical approach | Hurt |
| redistribute responders when incident rates change instead of after every incident | Emergency responder stationing · Emergency responder stationing | average response time | Helped |
Surprising findings
- A centralized (single-region) learned variant performs substantially worse than the hierarchical version.
Limitations
- Low-level training takes about 1 day to up to 14 days per region depending on depot count.
- Evaluated only on Nashville in the main text; Seattle results deferred to appendix.
- Response-time gains over MCTS are modest (5 to 13 seconds).
Experimental setup
Datasets: Nashville emergency response incident chains Seattle emergency response data
Benchmarks: Nashville incident chains (10 held-out) Seattle incident data
Metrics: average response time (seconds) computation time per decision
Baselines: MCTS DRLSN p-median policy greedy policy static policy GTrXL LSTM
Builds on: DDPG Transformer-XL hierarchical MCTS repositioning framework (Sivagnanam et al. / Pettet et al.) DRLSN
Closest papers by meaning
- One Step is Enough: Multi-Agent Reinforcement Learning Based on One-Step Policy Optimization for Order Dispatch on Ride-Sharing Platforms (NeurIPS 2026 · similarity 0.84)
- Rethinking Decision Transformer via Hierarchical Reinforcement Learning (ICML 2024 · similarity 0.83)
- ASP-Driven Emergency Planning for Norm Violations in Reinforcement Learning (AAAI 2025 · similarity 0.82)
- LNS2+RL: Combining Multi-agent Reinforcement Learning with Large Neighborhood Search in Multi-agent Path Finding (AAAI 2025 · similarity 0.82)
- Sim911: Towards Effective and Equitable 9-1-1 Dispatcher Training with an LLM-Enabled Simulation (AAAI 2025 · similarity 0.82)
- Semi-Markovian Planning to Coordinate Aerial and Maritime Medical Evacuation Platforms (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.