Research map / Optimization / Nonconvex and smooth optimization
Non-convex entropic mean-field optimization via Best Response flow
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that the Best Response (fictitious play) flow converges exponentially in W1 to the unique global minimizer of entropy-regularized non-convex mean-field functionals (and MNE of non-convex-non-concave min-max games) when the regularization is large enough relative to non-convexity and the reference measure's first moment.
Problem
Minimizing non-convex functionals over probability measures regularized by KL divergence to a reference measure, and entropy-regularized non-convex-non-concave min-max problems, e.g. mean-field softmax policy optimization in MDPs and Markov games, where global convergence of gradient flows is not gu
Approach
Shows the Best Response operator is a W1-contraction when sigma exceeds a bound linking the Lipschitz/boundedness constants of the flat derivative to the first moment of the reference measure, so the BR flow converges exponentially to the unique global minimizer without requiring convexity of F; extends to min-max games with distinct regularizers and learning rates.
Key results
- Best Response operator is W1-Lipschitz for any sigma>0 and a W1-contraction when sigma > 2C_F + e(e+1)L_F * (first moment of reference measure), giving a unique global minimizer.
- Stability estimate: W1(nu_t, nu_sigma*) decays as exp(-alpha t (1 - L)) under the contraction condition, yielding exponential convergence of the BR flow.
- Extends the contraction and convergence to min-max games with distinct regularizers and learning rates per player (sufficient sigma_nu, sigma_mu conditions).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use sufficiently large entropy regularization, with the required threshold linked to the non-convexity bound and reference-measure first moment | Best Response flow for single-agent entropy-regularized optimization · Single-agent entropy-regularized optimization | Wasserstein convergence: e^{-\alpha t\left(1-L_{\Psi_{\sigma},U^{\xi}} \right) } | Helped |
| Use sufficiently large regularization parameters for both players | Joint Best Response flow · Entropy-regularized non-convex-non-concave min-max | Wasserstein convergence: e^{-t\left(\min\{\alpha_\nu,\alpha_\mu\}-\left(\alpha_\nu L_{\Psi_{\sigma_\nu},U^{\xi}} + \alpha_\mu L_{\Phi_{\sigma_\mu | Helped |
| Choose a reference measure with a smaller first moment (lighter tails) | Single-agent Best Response operator · Single-agent entropy-regularized optimization | Wasserstein contraction | Helped |
| Allow different regularization parameters and learning rates for each player | Joint Best Response flow · Entropy-regularized non-convex-non-concave min-max | Wasserstein convergence | Helped |
Limitations
- Contraction requires sigma large relative to C_F, L_F and the first moment of the reference measure, a sufficient but possibly restrictive condition.
- Discrete-time implementation and interaction of sigma with step sizes is not analyzed; numerical scheme is only described.
- No experiments are included; results are purely theoretical.
Experimental setup
Metrics: W1 distance convergence rate
Baselines: Wasserstein gradient flow (Kim et al.) Best Response flow convex-concave min-max analysis
Builds on: Best Response flow / fictitious play flow Wasserstein gradient flow for mean-field Langevin Fisher-Rao gradient flow
Closest papers by meaning
- Convergence Theorems for Entropy-Regularized and Distributional Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
- State Entropy Regularization for Robust Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
- Global linear convergence of entropy-regularized softmax policy gradient beyond tabular MDPs (NeurIPS 2026 · similarity 0.86)
- A Reduction from Delayed to Immediate Feedback for Online Convex Optimization with Improved Guarantees (NeurIPS 2026 · similarity 0.86)
- Symmetric Mean-field Langevin Dynamics for Distributional Minimax Problems (ICLR 2024 · similarity 0.85)
- Flow Matching Policy Optimization with Mirror Descent and Entropy Constraints (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.