Research map / Reinforcement learning / Policy gradient and actor-critic
Learning values across many orders of magnitude
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace reward and TD-error clipping with Pop-Art | Double DQN · Atari 2600 (57 games) | normalized score difference vs clipped Double DQN: median (+0.4%) and mean (+34%) differences; 32 out of 57 games performance is at least as good | Mixed |
| Use Pop-Art instead of clipping | Double DQN with unclipped rewards · Atari 2600 (across games) | range of 95% of median gradient norms: roughly between 1 and 20; less than two orders of magnitude, compared to almost four orders of magnitude for clipped Dou | Helped |
Closest papers by meaning
- Natural Value Approximators: Learning when to Trust Past Estimates (NeurIPS 2017 · similarity 0.90)
- Orchestrated Value Mapping for Reinforcement Learning (ICLR 2022 · similarity 0.89)
- Penalizing Infeasible Actions and Reward Scaling in Reinforcement Learning with Offline Data (ICML 2025 · similarity 0.87)
- Learning the Target Network in Function Space (ICML 2024 · similarity 0.86)
- A Computationally Efficient Algorithm for Infinite-Horizon Average-Reward Linear MDPs (ICML 2025 · similarity 0.85)
- Investigating Pre-Training Objectives for Generalization in Vision-Based Reinforcement Learning (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.