Research map / Reinforcement learning / Policy gradient and actor-critic
A Generalized Bootstrap Target for Value-Learning, Efficiently Combining Value and Feature Predictions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use an intermediate η-return mixture (η=0.7) instead of TD(0) or full successor-feature bootstrapping | Value prediction in the 16-state deterministic chain · 16-state deterministic chain | speed of value estimation | Helped |
| Use an intermediate η rather than either endpoint (ablation) | η-return mixture · 19-state random chain | value error | Helped |
| Use an intermediate η rather than η=0 or η=1 (ablation) | Deep η-Q · Mini-Atari | average return | Helped |
| Set η=1, relying entirely on the full successor-feature value (ablation) | Deep η-Q · Mini-Atari | average return | Hurt |
| Use η=0, the value-only bootstrap target (ablation) | Deep η-Q · Mini-Atari | average return | Mixed |
| Increase η toward high values (ablation) | Deep η-Q · Mini-Atari freeway | average return | Hurt |
| Vary the value-head and encoder learning rates away from the recommended setting (ablation) | Vanilla DQN · Mini-Atari | average return | Hurt |
| Vary the successor-feature and reward-model learning rates while keeping the encoder and value-head rates fixed (ablation) | Deep η-Q · Mini-Atari | average return | No clear effect |
| Set the successor-feature learning rate higher than the representation-torso learning rate (ablation) | Deep η-Q · Mini-Atari freeway | average return | Helped |
Closest papers by meaning
- Target Networks and Over-parameterization Stabilize Off-policy Bootstrapping with Function Approximation (ICML 2024 · similarity 0.89)
- Foresee then Evaluate: Decomposing Value Estimation with Latent Future Prediction (AAAI 2021 · similarity 0.88)
- Discerning Temporal Difference Learning (AAAI 2024 · similarity 0.88)
- Long-Horizon Q-Learning: Accurate Value Learning via n-Step Inequalities (NeurIPS 2026 · similarity 0.87)
- Horizon Adaptive Offline Policy Learning via Value Stitching (NeurIPS 2026 · similarity 0.87)
- Flattening Hierarchies with Policy Bootstrapping (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.