Research map / Optimization / Nonconvex and smooth optimization
Non-Asymptotic Guarantees for Average-Reward Q-Learning with Adaptive Stepsizes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Gives the first finite-time analysis of asynchronous average-reward Q-learning with adaptive per-state-action stepsizes, showing mean-square error in span seminorm converges at about O(1/k).
Problem
Average-reward Q-learning lacks finite-time convergence guarantees, especially in the asynchronous setting where only one state-action pair is updated per step.
Approach
Uses adaptive stepsizes that act as local clocks for each state-action pair in asynchronous average-reward Q-learning, and analyzes the mean-square error in the span seminorm.
Key results
- Mean-square error of asynchronous average-reward Q-learning with adaptive stepsizes, measured in span seminorm, converges at a rate of about O(1/k) per the abstract.
Experimental setup
Metrics: mean-square error in span seminorm
Builds on: Q-learning average-reward reinforcement learning
Closest papers by meaning
- Sample Complexity of Asynchronous Q-Learning: Sharper Analysis and Variance Reduction (NeurIPS 2020 · similarity 0.91)
- Linear $Q$-Learning Does Not Diverge in $L^2$: Convergence Rates to a Bounded Set (ICML 2025 · similarity 0.87)
- Quotient-Categorical Representations for Bellman-Compatible Average-Reward Distributional Reinforcement Learning (NeurIPS 2026 · similarity 0.87)
- Sample Efficient Reinforcement Learning with Partial Dynamics Knowledge (AAAI 2024 · similarity 0.87)
- Finite-Time Bounds for Average-Reward Fitted Q-Iteration (NeurIPS 2025 · similarity 0.87)
- Optimal Non-Asymptotic Rates of Value Iteration for Average-Reward Markov Decision Processes (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.