AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

Log-normality and Skewness of Estimated State/Action Values in Reinforcement Learning

Liangpeng Zhang, Ke Tang, Xin Yao

NeurIPS 2017 · Reinforcement learning › Policy gradient and actor-critic · approach: Temporal-difference and discounting theory · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
increase observations per state-action from 20 to 200value iteration estimator · Chain MDPprobability of underestimation: 0.741 -> 0.603Improved
increase observations per state-action from 200 to 400value iteration estimator · Chain MDPprobability of underestimation: 0.603 -> 0.563Improved
increase observations per state-action from 10 to 50value iteration estimator · Complex mazeprobability of underestimation: 0.676 -> 0.597Improved
increase observations per state-action from 100 to 200value iteration estimator · Complex mazeprobability of underestimation: 0.563 -> 0.556No clear effect
use policy with positive-only immediate rewards (pi+) versus deterministic pi-value iteration estimator · Chain MDPprobability of underestimation: 0.603Hurt
collect more data to push underestimation below asymptotevalue iteration estimator · Complex mazeprobability of underestimation: 0.5546No clear effect
test log-normal theory against empirical estimatesTheorem 3.7 log-normal distribution · Chain MDPKS test p-value: 0.1190No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy