Research map / Multi-agent systems and games / Cooperative multi-agent RL

Multi-Agent Reinforcement Learning Meets Leaf Sequencing in Radiotherapy

ICML 2024 · Multi-agent systems and games › Cooperative multi-agent RL · approach: MARL benchmarks and grouping · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Reinforced Leaf Sequencer (RLS), a two-level multi-agent PPO model that sequences MLC leaf positions and monitor units for radiotherapy planning, evaluated against Varian's PORIx optimizer on head-and-neck and prostate data.

Problem

Leaf sequencing in radiotherapy planning is dominated by slow iterative optimization methods, and its serial, large output space makes it hard for supervised learning and end-to-end AI pipelines.

Approach

Formulates leaf sequencing as a finite-horizon multi-agent RL problem, with one shared leaf policy agent per leaf pair and a separate MU agent per control point, trained with PPO and five designed reward components so the model outputs executable sequences in a single pass per control point, with user-tunable reward weights controlling movement patterns.

Key results

• RLS reduced MNSE versus PORIx on all sites, e.g., HNd 0.149 vs 0.219 and Pros 0.042 vs 0.079.
• RIRE97% on HNd dropped from 18.8 (PORIx) to 12.6 (RLS), and on HNe1 from 26.3 to 10.7.
• In the full-AI VMAT prostate pipeline, RLS achieved Dose MAE 0.19 and DVH MAE 0.88, and IMRT simulation achieved Dose MAE 0.23.

Measured effects reported in the paper

Surprising findings

• Training on 50 patients (MNSE 0.158) remains close to training on 370 patients (0.149), suggesting limited sensitivity to training size.
• Sharing the backbone between actor and critic is only slightly worse than separate networks, and PPG offers minor gains, so the backbone choice matters little.

Limitations

• Comparison is limited to PORIx because few comparable open-source leaf sequencers exist.
• Only the leaf sequencing module is trained, so global plan quality in PORIx may not be fully optimized.
• Evaluation covers only the first 100 PORIx iterations and a preliminary end-to-end pipeline, not clinical plan quality.

Experimental setup

Datasets: HNd (in-house head-and-neck) HNe1 (TCIA HNSCC-3DCT-RT) HNe2 (TCIA HNSCC) Pros (public prostate, permission required)

Benchmarks: HNd HNe1 HNe2 Pros Pros(e) OpenKBP

Metrics: MNSE RIRE97% AIRE97% AIRE99% Dose score DVH score

Baselines: PORIx leaf sequencer OpenKBP top entries

Builds on: PPO CleanRL GAE PPG

Closest papers by meaning

• Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progress (NeurIPS 2022 · similarity 0.83)
• Dr. MAS: Stable Reinforcement Learning for Multi-Agent LLM Systems (NeurIPS 2026 · similarity 0.82)
• RL-GPT: Integrating Reinforcement Learning and Code-as-policy (NeurIPS 2024 · similarity 0.82)
• UFO-RL: Uncertainty-Focused Optimization for Efficient Reinforcement Learning Data Selection (NeurIPS 2025 · similarity 0.82)
• Task-Agnostic Pre-training and Task-Guided Fine-tuning for Versatile Diffusion Planner (ICML 2025 · similarity 0.82)
• Curriculum Design for Trajectory-Constrained Agent: Compressing Chain-of-Thought Tokens in LLMs (NeurIPS 2025 · similarity 0.81)