Research map / Reinforcement learning / Policy gradient and actor-critic
Single-Timescale Actor-Critic Provably Finds Globally Optimal Policy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Update actor and critic simultaneously, using one Bellman evaluation update per iteration | Actor-critic with linear function approximation · Discounted MDPs | cumulative regret: O (K^{1/2}) | Helped |
| Use overparameterized deep neural networks for both actor and critic in single-timescale updates | Single-timescale actor-critic · Discounted MDPs | cumulative regret: O (K^{1/2}) | Helped |
| Reuse a fixed off-policy dataset instead of drawing new data for each critic update | Linear single-timescale actor-critic · ε-globally optimal policy | sample complexity: \tilde O(\varepsilon^{-4}) -> \tilde O(\varepsilon^{-2}) | Helped |
| Use off-policy data reuse with the deep neural actor-critic method | Single-timescale deep neural actor-critic · ε-globally optimal policy | sample complexity: \tilde O(\varepsilon^{-14}) -> \tilde O(\varepsilon^{-12}) | Helped |
Closest papers by meaning
- On the Convergence of Single-Timescale Actor-Critic (NeurIPS 2025 · similarity 0.92)
- Non-Asymptotic Analysis for Single-Loop (Natural) Actor-Critic with Compatible Function Approximation (ICML 2024 · similarity 0.91)
- Finite-Time Global Optimality Convergence in Deep Neural Actor-Critic Methods for Decentralized Multi-Agent Reinforcement Learning (ICML 2025 · similarity 0.91)
- On the Convergence of Continuous Single-timescale Actor-critic (ICML 2025 · similarity 0.90)
- Two-Timescale Critic-Actor for Average Reward MDPs with Function Approximation (AAAI 2025 · similarity 0.90)
- Closing the Gap: Achieving Global Convergence (Last Iterate) of Actor-Critic under Markovian Sampling with Neural Network Parametrization (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.