Research map / Reinforcement learning / Visual and representation RL

Improving Autonomous Separation Assurance through Distributed Reinforcement Learning with Attention Networks

AAAI 2024 · Reinforcement learning › Visual and representation RL · approach: RL for combinatorial routing · 7 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SACD-A, a decentralized discrete soft actor-critic with attention networks and asynchronous distributed training for aircraft separation assurance via speed and vertical maneuvers in AAM corridors, reporting lower risk ratio than D2MAV-A and ~10x more training transitions.

Problem

Autonomous self-separation of many low-altitude aircraft in dense Advanced Air Mobility corridors with variable numbers of intruders, sensing uncertainty, and real-time safety-critical decisions, where conventional separation assurance and fixed-length-state DRL approaches scale poorly.

Approach

Extend discrete soft actor-critic with an attention network that compresses a variable number of intruder states into a fixed-length vector, add vertical maneuvers to the action set, and decouple off-policy training from distributed actor rollouts for high sample throughput, with target entropy annealing.

Key results

• SACD-A achieves a risk ratio of 0.05 when at least 95% of aircraft are equipped with the policy, and outperforms D2MAV-A across fleet sizes and communication probabilities.
• At probability of equipage 0.75 or below, D2MAV-A outperforms SACD-A; SACD-A is better above 0.75.
• SACD-A trained on 2.36 billion transitions versus 256 million for D2MAV-A (~10x).

Measured effects reported in the paper

Surprising findings

• D2MAV-A outperforms SACD-A when probability of equipage is at or below 0.75, suggesting better adaptation to unequipped aircraft.

Limitations

• Simulation-only evaluation without action latency or weather models, as acknowledged in the discussion.
• Underperforms D2MAV-A when few aircraft are equipped (probability of equipage at or below 0.75).
• Requires further simulation and human-in-the-loop studies before real-world deployment.

Experimental setup

Datasets: AAM traffic scenarios generated by UAMToolkit over 20 days, New York City VFR route network

Benchmarks: AAM-Gym simulated NYC urban air corridor scenarios (100 sampled 3-hour windows)

Metrics: risk ratio (NMAC probability relative to unequipped) action distribution (operational suitability) training transitions

Baselines: D2MAV-A unequipped agent (no separation logic)

Builds on: Discrete Soft Actor-Critic D2MAV-A Proximal Policy Optimization AAM-Gym

Closest papers by meaning

• One Step is Enough: Multi-Agent Reinforcement Learning Based on One-Step Policy Optimization for Order Dispatch on Ride-Sharing Platforms (NeurIPS 2026 · similarity 0.84)
• UAV-Flow Colosseo: A Real-World Benchmark for Flying-on-a-Word UAV Imitation Learning (NeurIPS 2025 · similarity 0.83)
• NeuralPlane: An Efficiently Parallelizable Platform for Fixed-wing Aircraft Control with Reinforcement Learning (NeurIPS 2024 · similarity 0.82)
• AIR: Unifying Individual and Collective Exploration in Cooperative Multi-Agent Reinforcement Learning (AAAI 2025 · similarity 0.82)
• HMARL-CBF – Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systems (NeurIPS 2025 · similarity 0.81)
• Dyna-Style Safety Augmented Reinforcement Learning: Staying Safe in the Face of Uncertainty (NeurIPS 2026 · similarity 0.81)