Research map / Multi-agent systems and games / Games and equilibria
Learning to Stop: Deep Learning for Mean Field Optimal Stopping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formulates discrete-time, finite-space mean-field optimal stopping (MFOS), proves a dynamic programming principle and O(1/sqrt N) approximation to the N-agent problem, and proposes two neural methods (Direct Approach and DP backward induction) evaluated on six environments.
Problem
Multi-agent optimal stopping becomes computationally prohibitive as the number of agents grows; no mean-field or computational methods existed for discrete-time, finite-space mean-field optimal stopping.
Approach
Extend optimal stopping to a mean-field control setting with randomized stopping times and an extended "alive" state, prove a DPP for the mean-field problem, and learn state- and distribution-dependent stopping probabilities either by directly minimizing the social cost through trajectories (DA) or by backward induction using the DPP (DP).
Key results
- Empirical gap between N-agent and mean-field cost and L2 distribution distance decay at rate about N^{-1/2} on Example 1 (averaged over 10 runs).
- On Example 1 the learned policy approaches the analytical optimal value (T+2)/(2(T+1)), recovered in a few iterations.
- Both DA and DP solve six environments with state spaces up to 10x10 grid (input dimension 300), with each run taking at most 3 minutes on GPU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Allow randomized stopping instead of restricting the policy to the non-randomized MF strategy | N-agent optimal stopping in the Randomized is better example · Randomized is better example | social cost: 4 -> 2 | Helped |
| Use asynchronous rather than synchronous stopping times | DA · Example 3: Crowd Motion with Congestion | social cost | Helped |
| Use DA rather than DP | MFOS neural-network training · MFOS experiments | training speed | Helped |
| Use DP rather than DA | MFOS neural-network training · MFOS problems with long time horizon | memory requirement | Helped |
Surprising findings
- Deterministic (non-randomized) mean-field stopping can be strictly suboptimal for the N-agent problem, motivating randomized stopping times.
Limitations
- No convergence proof for the algorithms due to difficulty of analyzing deep networks.
- Restricted to finite state space and discrete time; continuous space extension left to future work.
- DA memory grows with horizon T while DP needs constant memory per step; no detailed analysis of different classes of stopping times.
Experimental setup
Metrics: social cost L2 distance of empirical to mean-field distribution optimality gap between N-agent and mean-field cost
Builds on: mean-field control theory dynamic programming principle least-square Monte Carlo
Closest papers by meaning
- Solving Continuous Mean Field Games: Deep Reinforcement Learning for Non-Stationary Dynamics (NeurIPS 2025 · similarity 0.86)
- Mean Field Langevin Actor-Critic: Faster Convergence and Global Optimality beyond Lazy Learning (ICML 2024 · similarity 0.85)
- Scalable Neural Incentive Design with Parameterized Mean-Field Approximation (NeurIPS 2025 · similarity 0.85)
- Deep learning for continuous-time stochastic control with jumps (NeurIPS 2025 · similarity 0.85)
- Bench-MFG: A Benchmark Suite for Learning in Stationary Mean Field Games (NeurIPS 2026 · similarity 0.84)
- The Courage to Stop: Overcoming Sunk Cost Fallacy in Deep Reinforcement Learning (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.