Research map / Reinforcement learning / Policy gradient and actor-critic
Relative Value Learning
Marc Höftmann, Jan Robine, Stefan Harmeling
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace absolute value critic with relative value critic (RV) in PPO | PPO · Atari 49 games | games with higher mean score: exceeds PPO on 30 of 49 games (61%) | Mixed |
| replace absolute value critic with relative value critic (RV) in PPO | PPO · Atari 49 games vs DAE | games with higher mean score: exceeds DAE on 37 of 49 games (75%) | Improved |
| RV critic on Enduro | PPO · Atari, Enduro | mean final score: 758.3 -> 1080.2 | Improved |
| RV critic on VideoPinball | PPO · Atari, VideoPinball | mean final score: 37389.0 -> 138564.8 | Improved |
| RV critic on Zaxxon | PPO · Atari, Zaxxon | mean final score: 5008.7 -> 845.8 | Hurt |
| RV critic on Centipede | PPO · Atari, Centipede | mean final score: 4386.4 -> 1226.1 | Hurt |
| add trajectory-ranking offset initialization instead of zero initialization of relative values (ablation) | PPO+RV · Atari (ablation figure) | score | Mixed |
| use non-antisymmetric function gamma*V(s')-V(s) instead of antisymmetric R-GAE (ablation) | PPO+RV · Atari | score | Hurt |
Closest papers by meaning
- The Value-Improvement Path: Towards Better Representations for Reinforcement Learning (AAAI 2021 · similarity 0.89)
- Natural Value Approximators: Learning when to Trust Past Estimates (NeurIPS 2017 · similarity 0.88)
- Pretrain Value, Not Reward: Decoupled Value Policy Optimization (ICLR 2026 · similarity 0.87)
- A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning (AAAI 2024 · similarity 0.87)
- Enhancing Value Function Estimation through First-Order State-Action Dynamics in Offline Reinforcement Learning (ICML 2024 · similarity 0.86)
- Is Value Learning Really the Main Bottleneck in Offline RL? (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
