Research map / Reinforcement learning / Policy gradient and actor-critic
Hindsight Credit Assignment
Anna Harutyunyan, Will Dabney, Thomas Mesnard, Mohammad Gheshlaghi Azar, Bilal Piot, Nicolas Heess, Hado P van Hasselt, Gregory Wayne, Satinder Singh, Doina Precup, Remi Munos
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use state-conditioned hindsight credit assignment instead of baseline policy gradient | Policy gradient · Shortcut | learning performance | Improved |
| Use state-conditioned hindsight credit assignment instead of naive bootstrapping | Policy gradient · Delayed Effect | learning performance | Improved |
| Use state-conditioned hindsight credit assignment instead of Monte Carlo returns | Policy gradient · Delayed Effect with noisy intermediate rewards | performance | Improved |
| Use state-conditioned hindsight credit assignment instead of baseline policy gradient | Policy gradient · Ambiguous Bandit with observed state identity | average performance | Improved |
| Use return-conditioned hindsight credit assignment instead of baseline policy gradient | Policy gradient · Ambiguous Bandit with hidden state identity | average performance | Improved |
| Use state-conditioned hindsight credit assignment with hidden state identity | Policy gradient · Ambiguous Bandit with hidden state identity | average performance | Hurt |
| Vary the number of bins for return-conditioned HCA (ablation) | Return-conditioned HCA · Experiments | performance | Mixed |
Closest papers by meaning
- Hindsight PRIORs for Reward Learning from Human Preferences (ICLR 2024 · similarity 0.86)
- From Past to Future: Rethinking Eligibility Traces (AAAI 2024 · similarity 0.86)
- Provably Efficient Partially Observable Risk-sensitive Reinforcement Learning with Hindsight Observation (ICML 2024 · similarity 0.85)
- Prospective Hindsight: Self-Calibrating Reinforcement Learning via Prediction–Reality Gaps (NeurIPS 2026 · similarity 0.85)
- Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learning (AAAI 2025 · similarity 0.85)
- Sequence Compression Speeds Up Credit Assignment in Reinforcement Learning (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
