Research map / Reinforcement learning / Policy gradient and actor-critic
Doubly-Asynchronous Value Iteration: Making Value Iteration Asynchronous in Actions
Tian Tian, Kenny John Young, Richard S. Sutton
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DAVI with sampled action subsets instead of Asynchronous VI (full action maximization) | DAVI · Single-state needle-in-the-haystack (10000 actions, 1 rewarding) | state value vs computation | Mixed |
| use DAVI instead of Asynchronous VI when several actions are rewarding | DAVI · Single-state multi-reward (10 of 10000 actions rewarding) | state value vs computation | Improved |
| use DAVI instead of Asynchronous VI and VI on a tree-structured MDP with one rewarding leaf | DAVI · Tree MDP (depth 2, 50 actions/state, ~10000 states) | state value vs computation | No clear effect |
| use DAVI instead of Asynchronous VI and VI on a random MDP with multiple paths to the reward | DAVI · Random MDP (100 states, 1000 actions) | state value vs computation | Improved |
| increase action subset size m from 1 to 10, 100, 1000 (ablation) | DAVI · Single-state experiments | convergence rate | Improved |
| increase action subset size m from 10 to 100 or 1000 (ablation) | DAVI · Single-state experiments | convergence rate | No clear effect |
| set action subset size m to 10 rather than other values (ablation) | DAVI · Random MDP (100 states, 1000 actions) | convergence rate | Improved |
Closest papers by meaning
- Sketched Newton Value Iteration for Large-Scale Markov Decision Processes (AAAI 2024 · similarity 0.90)
- Revisiting Value Iteration: Unified Analysis of Discounted and Average-Reward Cases (NeurIPS 2026 · similarity 0.88)
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.88)
- Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training (NeurIPS 2025 · similarity 0.87)
- Exploring Starts Are Not Enough: Counterexamples and a Fix for Monte Carlo Exploring Starts (NeurIPS 2026 · similarity 0.87)
- Horizon Adaptive Offline Policy Learning via Value Stitching (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
