Research map / Reinforcement learning / Policy gradient and actor-critic
The Value-Improvement Path: Towards Better Representations for Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add PastPolicies auxiliary tasks using past policies in the improvement trajectory | Double DQN · Atari-57 | mean human-normalized score: approximately double | Helped |
| Add PastPolicies auxiliary tasks using past policies in the improvement trajectory | Double DQN · Atari-57 | median human-normalized score: approximately double | Helped |
| Add PastMixtures auxiliary tasks that learn different mixtures of past and current value estimates | Double DQN · Atari-57 | long-term performance | Helped |
| Add CumulantValue auxiliary tasks | Double DQN · Atari-57 | long-term performance | Hurt |
| Add CumulantValue auxiliary tasks | Double DQN · Atari-57 games with particularly difficult exploration | performance | Helped |
| Add PastMixtures auxiliary task | Rainbow · Atari-57 | median human-normalized score | No clear effect |
| Add PastMixtures auxiliary task | Rainbow · Atari-57 | mean human-normalized score | Helped |
| Add PastMixtures auxiliary task | Rainbow · Atari-57 | human-gap | Helped |
| Use past policies' value functions as auxiliary tasks | Double DQN · Atari-57 future performance vs. future-value generalization error | Pearson coefficient: -0.995 | Mixed |
| Use PastMixtures auxiliary tasks | Double DQN · Atari-57 future performance vs. future-value generalization error | Pearson coefficient: -0.895 | Mixed |
Closest papers by meaning
- Value Function Decomposition for Iterative Design of Reinforcement Learning Agents (NeurIPS 2022 · similarity 0.90)
- The Value Equivalence Principle for Model-Based Reinforcement Learning (NeurIPS 2020 · similarity 0.90)
- Is Value Learning Really the Main Bottleneck in Offline RL? (NeurIPS 2024 · similarity 0.89)
- Your Language Model is Its Own Critic: Reinforcement Learning with Value Estimation from Actor’s Internal States (NeurIPS 2026 · similarity 0.88)
- $\mathcal{B}$-Coder: Value-Based Deep Reinforcement Learning for Program Synthesis (ICLR 2024 · similarity 0.88)
- Stop Regressing: Training Value Functions via Classification for Scalable Deep RL (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.