AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

Doubly-Asynchronous Value Iteration: Making Value Iteration Asynchronous in Actions

Tian Tian, Kenny John Young, Richard S. Sutton

NeurIPS 2022 · Reinforcement learning › Policy gradient and actor-critic · approach: Temporal-difference and discounting theory · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use DAVI with sampled action subsets instead of Asynchronous VI (full action maximization)DAVI · Single-state needle-in-the-haystack (10000 actions, 1 rewarding)state value vs computationMixed
use DAVI instead of Asynchronous VI when several actions are rewardingDAVI · Single-state multi-reward (10 of 10000 actions rewarding)state value vs computationImproved
use DAVI instead of Asynchronous VI and VI on a tree-structured MDP with one rewarding leafDAVI · Tree MDP (depth 2, 50 actions/state, ~10000 states)state value vs computationNo clear effect
use DAVI instead of Asynchronous VI and VI on a random MDP with multiple paths to the rewardDAVI · Random MDP (100 states, 1000 actions)state value vs computationImproved
increase action subset size m from 1 to 10, 100, 1000 (ablation)DAVI · Single-state experimentsconvergence rateImproved
increase action subset size m from 10 to 100 or 1000 (ablation)DAVI · Single-state experimentsconvergence rateNo clear effect
set action subset size m to 10 rather than other values (ablation)DAVI · Random MDP (100 states, 1000 actions)convergence rateImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy