Research map / Reinforcement learning / Policy gradient and actor-critic
Learning State Representations from Random Deep Action-conditional Predictions
Zeyu Zheng, Vivek Veeriah, Risto Vuorio, Richard Lewis, Satinder Singh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use random deep action-conditional GVFs as auxiliary tasks | A2C · 49 Atari games | median human-normalized score | Improved |
| Use random deep action-conditional GVFs as auxiliary tasks | A2C · 12 DeepMind Lab environments | mean capped human-normalized score | Improved |
| Train representations solely with rGVF auxiliary tasks, stopping RL gradients to the representation | A2C · 49 Atari games | median human-normalized score | Improved |
| Train representations solely with rGVF auxiliary tasks, stopping RL gradients to the representation | A2C · 12 DeepMind Lab environments | mean capped human-normalized score | Improved |
| Add depth to action-conditional prediction targets (ablation) | rGVFs · Breakout | game score | Improved |
| Use deep action-conditional predictions instead of shallow predictions (ablation) | rGVFs · Six Atari games | episode return | Improved |
| Add action conditioning to random GVFs (ablation) | rGVFs · Six Atari games | episode return | Improved |
| Use deep action-conditional predictions instead of discounted-sum predictions (ablation) | rGVFs · Six Atari games | episode return | Mixed |
| Increase the number of random features (ablation) | Random-GVF representation learning · 7×7 grid world | MSE | Improved |
| Increase action-conditional tree depth to 3 (ablation) | Grid-world value-function learning · 7×7 grid world | MSE: MSE of 0 at depth 3 | Improved |
| Use random features instead of the hand-crafted touch feature (ablation) | Random-GVF representation learning · 7×7 grid world | MSE | Hurt |
| Use CURL auxiliary learning | A2C · 49 Atari games | human-normalized score | Hurt |
Closest papers by meaning
- Learning Value Functions in Deep Policy Gradients using Residual Variance (ICLR 2021 · similarity 0.88)
- Return-Based Contrastive Representation Learning for Reinforcement Learning (ICLR 2021 · similarity 0.88)
- Stochastic Q-learning for Large Discrete Action Spaces (ICML 2024 · similarity 0.87)
- Representation Learning Enables Scalable Multitask Deep Reinforcement Learning (NeurIPS 2026 · similarity 0.87)
- Efficient Off-Policy Learning for High-Dimensional Action Spaces (ICLR 2025 · similarity 0.86)
- Bridging State and History Representations: Understanding Self-Predictive RL (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
