Research map / Bandits and online learning / RL theory and MDPs
Stabilizing Q-learning with Linear Architectures for Provable Efficient Learning
Andrea Zanette, Martin Wainwright
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add policy replay to keep Q-learning updates on data from a stationary mixture of past policies | Sequentially Stabilized Second-order Streaming Q-learning · Exploratory learning in low-rank MDPs | regret | Improved |
| Add a second-order update instead of a first-order update | Stabilized, Second-Order, Streaming Q-learning · Stationary-controller learning | statistical efficiency: a rate $1/\sqrt{n}$ instead of the $1/n$ | Improved |
| Use target networks to hold the bootstrapped target fixed during each update epoch | Stabilized, Second-Order, Streaming Q-learning · Linear function approximation | stability | Improved |
| Remove policy replay and update using a changing data distribution | Sequentially Stabilized Second-order Streaming Q-learning · Exploratory learning with linear function approximation | stability | Hurt |
| Use policy replay rather than storing and replaying the full transition dataset | Sequentially Stabilized Second-order Streaming Q-learning · Exploratory learning | memory complexity: grows only logarithmically with sample size | Improved |
Closest papers by meaning
- Regularized Q-Learning (NeurIPS 2024 · similarity 0.89)
- A Nearly Optimal and Low-Switching Algorithm for Reinforcement Learning with General Function Approximation (NeurIPS 2024 · similarity 0.89)
- Linear $Q$-Learning Does Not Diverge in $L^2$: Convergence Rates to a Bounded Set (ICML 2025 · similarity 0.89)
- Sample Efficient Reinforcement Learning with Partial Dynamics Knowledge (AAAI 2024 · similarity 0.89)
- Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities (NeurIPS 2026 · similarity 0.88)
- Achieving Constant Regret in Linear Markov Decision Processes (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
