Research map / Multi-agent systems and games / Cooperative multi-agent RL

One Step is Enough: Multi-Agent Reinforcement Learning Based on One-Step Policy Optimization for Order Dispatch on Ride-Sharing Platforms

NeurIPS 2026 · Multi-agent systems and games › Cooperative multi-agent RL · approach: Cooperative MARL algorithms · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ST-GRPO and OSPO, critic-free multi-agent policy optimization for AV order dispatch that replaces value baselines with fleet-average rewards (ST-GRPO) or one-step group-normalized rewards (OSPO), using MLP policies on Manhattan and Queens ride-hailing data.

Problem

Order dispatch for ride-sharing fleets of autonomous vehicles requires MARL, but value-function (Q/V) estimation in large, highly stochastic urban environments is biased and unstable, hurting training efficiency and policy quality.

Approach

Exploit fleet homogeneity and short-memory mixing to show agent value functions share a common time-varying baseline, so advantages can be computed from the fleet-average reward-to-go (ST-GRPO) or, under bounded-value and mixing conditions, from immediate group-normalized rewards alone (OSPO), removing the critic entirely.

Key results

• On Manhattan, OSPO achieves the best reward (11.00 ± 2.63 ×10^3), service rate 1.00, and pickup time 6.71 min versus 8.67 for DeepPool and 8.10 for Hoppe et al.
• ST-GRPO achieves pickup time 7.33 and service rate 1.00 on Manhattan with a 20K-parameter MLP and 5.17 GB GPU occupation.
• On Queens, OSPO reports reward 5521.74 and service rate 0.67 versus 0.66 for BMG-Q, with pickup time 11.88 versus 12.30 for DeepPool.

Surprising findings

• OSPO, which uses only immediate group-normalized rewards with no long-horizon bootstrapping, outperforms ST-GRPO that uses fleet-averaged reward-to-go.
• MAPPO failed to converge due to the curse of dimensionality in its centralized critic, and only IPPO was competitive among the policy-gradient baselines.

Limitations

• Theoretical guarantees rely on assumptions (homogeneity, uniform beta-mixing, bounded V-values) that may not hold during training exploration; the authors add a heuristic regularization term to address this.
• Evaluation limited to two NYC datasets with a 30-minute episode horizon; results on Queens show ST-GRPO and OSPO gains are small in service rate.
• Reported differences are small in several metrics (e.g., service rate 0.64 vs 0.67 on Queens) and the paper's own analysis uses a finite-horizon approximation.

Experimental setup

Datasets: NYC High Volume For-Hire Vehicle trip records (Manhattan, July 2024) NYC FHV trip records (Queens, July 2024)

Benchmarks: Manhattan ride-hailing test set (24 half-hour episodes, July 18, 2024) Queens FHV dataset (July 17, 2024)

Metrics: reward service rate delivery time detour time pickup time confirmation time

Baselines: DeepPool BMG-Q HIVES Enders et al. (MASAC) CEVD Hoppe et al. (COMA) IPPO MAPPO IPG

Builds on: GRPO PPO IPPO MAPPO DeepPool

Closest papers by meaning

• GARLIC: GPT-Augmented Reinforcement Learning with Intelligent Control for Vehicle Dispatching (AAAI 2025 · similarity 0.89)
• NondBREM: Nondeterministic Offline Reinforcement Learning for Large-Scale Order Dispatching (AAAI 2024 · similarity 0.88)
• URB - Urban Routing Benchmark for RL-equipped Connected Autonomous Vehicles (NeurIPS 2025 · similarity 0.87)
• i-Rebalance: Personalized Vehicle Repositioning for Supply Demand Balance (AAAI 2024 · similarity 0.87)
• Hierarchical Optimization via LLM-Guided Objective Evolution for Mobility-on-Demand Systems (NeurIPS 2025 · similarity 0.87)
• Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuning (ICLR 2026 · similarity 0.86)