Research map / Reinforcement learning / Visual and representation RL
Hindsight policy gradients
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use hindsight policy gradients instead of goal-conditional policy gradients | Sparse-reward bit-flipping environment (k=8, batch size 16) · Bit flipping (k=8) | batches to obtain excellent policies: more than 400 batches earlier | Helped |
| Use hindsight policy gradients instead of goal-conditional policy gradients | Sparse-reward bit-flipping environment (k=16, batch size 16) · Bit flipping (k=16) | learning performance | Helped |
| Add an approximate value-function baseline to HPG | HPG · Bit flipping (k=8) | training stability | Hurt |
| Use hindsight policy gradients instead of goal-conditional policy gradients | Sparse-reward empty-room grid world · Empty room | batches to obtain excellent policies: at least 200 batches | Helped |
| Use hindsight policy gradients instead of goal-conditional policy gradients | Sparse-reward four-rooms grid world · Four rooms | sample efficiency | Helped |
| Use HPG instead of GCPG | Ms. Pac-man (batch size 16) · Ms. Pac-man | policy performance | Mixed |
| Use HPG instead of GCPG | Ms. Pac-man (small batch size, all active goals used) · Ms. Pac-man | policy performance | Helped |
| Use HPG instead of GCPG | FetchPush (batch size 16) · FetchPush | learning performance | Helped |
| Remove likelihood-ratio corrections from HPG (ablation) | HPG · Bit flipping | hyperparameter sensitivity | Hurt |
| Remove likelihood-ratio corrections from HPG (ablation) | HPG · Four rooms | hyperparameter sensitivity | Hurt |
| Add an approximate value-function baseline to HPG (ablation) | HPG · Grid world environments | training stability | Mixed |
| Subsample active goals during HPG training (ablation) | HPG · Ms. Pac-man (batch size 16) | final policy performance: 3 active goals per episode (out of potentially 28) | Hurt |
Closest papers by meaning
- Prospective Hindsight: Self-Calibrating Reinforcement Learning via Prediction–Reality Gaps (NeurIPS 2026 · similarity 0.89)
- Provably Efficient Partially Observable Risk-sensitive Reinforcement Learning with Hindsight Observation (ICML 2024 · similarity 0.88)
- Goal-Conditioned On-Policy Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Beyond Optimism: Exploration With Partially Observable Rewards (NeurIPS 2024 · similarity 0.87)
- PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabeling (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.