AI Paper Atlas

Research map / Reinforcement learning / Policy gradient and actor-critic

Using a Logarithmic Mapping to Enable Lower Discount Factors in Reinforcement Learning

Harm Van Seijen, Mehdi Fatemi, Arash Tavakoli

NeurIPS 2019 · Reinforcement learning › Policy gradient and actor-critic · approach: Temporal-difference and discounting theory · 10 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use logarithmic Q-learning instead of regular Q-learningTile-coded chain task with function approximation · Chain taskoptimal-policy performanceImproved
Increase reward magnitudes by a factor of 100 (ablation)Regular Q-learning · Chain taskoptimal-policy performanceNo clear effect
Add rewards that push action-values up by 100 (ablation)Regular Q-learning · Chain taskoptimal-policy performanceNo clear effect
Use logarithmic mapping for positive rewards only (ablation)Logarithmic Q-learning · Chain taskaction-gap deviationImproved
Use logarithmic mapping for negative rewards only (ablation)Logarithmic Q-learning · Chain taskaction-gap deviationMixed
Use logarithmic mapping for both positive and negative rewards (ablation)Logarithmic Q-learning · Chain taskaction-gap deviationMixed
Add a constant bias before applying the logarithm to handle negative rewards (ablation)Logarithmic Q-learning · Chain taskaction-gap deviationHurt
Set the regular-space averaging step size to 1, removing the separate regular-space averaging correction (ablation)Logarithmic Q-learning · Positive-reward chain taskRMS errorHurt
Use lower discount factors, from γ=0.84 to γ=0.99, with LogDQN (ablation)LogDQN · Six Atari games used for tuninghuman-normalized scoreNo clear effect
Use LogDQN rather than DQNDQN · 55 Atari gameshuman-normalized scoreImproved
Use Huber loss rather than squared-error loss (ablation)LogDQN · AtariperformanceNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy