Research map / Multi-agent systems and games / Cooperative multi-agent RL
Discrete GCBF Proximal Policy Optimization for Multi-agent Safe Optimal Control
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DGPPO, an on-policy MARL framework that jointly learns a discrete graph control barrier function (DGCBF), taken as the constraint-value function of the policy, and a distributed safe policy for multi-agent systems with unknown discrete-time dynamics, limited sensing and input constraints.
Problem
Distributed safe optimal control for multi-agent systems with unknown discrete-time dynamics, changing neighborhoods under limited sensing, input constraints, and no known performant nominal policy.
Approach
Shows that the constraint-value function of a policy is a discrete CBF, learns it with a GNN value function via deterministic rollouts, and enforces DCBF constraints via a CRPO-style primal update with score-function gradients and gradient projection, combined with PPO.
Key results
- DGPPO achieves the lowest cost among methods with near 100% safety rate using a single set of hyperparameters across environments.
- Using a hand-crafted DGCBF on Transport2 drops the safety rate to about 15%, supporting the need to learn the DGCBF.
- Using a stochastic rollout instead of deterministic rollout to learn the DGCBF degrades both cost and safety rate.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DGPPO with one fixed set of hyperparameters across environments | DGPPO · Target, Spread, Line, Bicycle, Transport, Transport2, Wheel | cost | Helped |
| use stochastic rather than deterministic rollouts to learn the DGCBF (ablation) | DGPPO · unspecified comparison | cost | Hurt |
| use stochastic rather than deterministic rollouts to learn the DGCBF (ablation) | DGPPO · unspecified comparison | safety rate | Hurt |
| use h^(m) directly as the DGCBF instead of the learned constraint-value function (ablation) | DGPPO · Transport2 | safety rate: ∼!15% | Hurt |
| remove the DGCBF (ablation) | DGPPO · Transport2 | safety rate | Hurt |
| set the class-κ slope to a=0 (ablation) | DGPPO · Spread | cost | Hurt |
| set the class-κ slope to a=1.2 (ablation) | DGPPO · Spread | safety rate | Hurt |
| vary the class-κ slope among other values satisfying the stated condition (ablation) | DGPPO · Spread | cost | No clear effect |
| vary the class-κ slope among other values satisfying the stated condition (ablation) | DGPPO · Spread | safety rate | No clear effect |
| use a static ν=1 instead of the proposed schedule (ablation) | DGPPO · Spread | safety rate | Hurt |
| use static ν=4 or ν=8 instead of the proposed schedule (ablation) | DGPPO · Spread | cost convergence speed | Hurt |
| use the proposed ν schedule instead of static schedules (ablation) | DGPPO · Spread | cost convergence speed | Helped |
Surprising findings
- Hand-crafted barrier functions used directly as DGCBFs yield only about 15% safety on Transport2.
Limitations
- Uses both stochastic and deterministic rollouts, reducing sample efficiency.
- Safety under stochastic dynamics not considered.
- Guarantee requires DGCBF constraints satisfied at all states, which is hard to achieve with learning.
Experimental setup
Benchmarks: Target Spread Line Bicycle Transport Transport2 Wheel
Metrics: cost safety rate
Baselines: InforMARL MAPPO-Lagrangian Penalty Schedule
Builds on: GCBF MAPPO CRPO InforMARL PPO
Closest papers by meaning
- Scalable Constrained Policy Optimization for Safe Multi-agent Reinforcement Learning (NeurIPS 2024 · similarity 0.87)
- HMARL-CBF – Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systems (NeurIPS 2025 · similarity 0.87)
- Scalable Safe Policy Improvement for Factored Multi-Agent MDPs (ICML 2024 · similarity 0.87)
- A Physics-Informed Machine Learning Framework for Safe and Optimal Control of Autonomous Systems (ICML 2025 · similarity 0.85)
- Learning Verified Safe Neural Network Controllers for Multi-Agent Path Finding (AAAI 2025 · similarity 0.85)
- Safe and Stable Control via Lyapunov-Guided Diffusion Models (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.