Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Forethought and Hindsight in Credit Assignment
Veronica Chelu, Doina Precup, Hado P van Hasselt
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| channelling structure (large fan-in, small fan-out) favors backward planning over forward planning (ablation) | backward planning · Random chain MRP, 3-level bipartite graph (n_x=500, n_y=5) | value prediction error | Improved |
| broadcasting structure (small fan-in, large fan-out, n_x=5, n_y=500) with backward instead of forward planning (ablation) | backward planning · Random chain MRP, 3-level bipartite graph | value prediction error | Hurt |
| plan backward from s instead of s' in full learning setting (ablation) | Online Backward-Dyna · discrete navigation task, full learning | learning efficiency | Improved |
| plan backward from s' instead of s in pure planning setting (ablation) | Online Backward-Dyna · discrete navigation task, pure planning | learning efficiency | Improved |
| use backward model instead of forward model under stochastic transitions (ablation) | Online Backward-Dyna · discrete navigation task, stochastic dynamics | credit assignment damage from model error | Improved |
| use backward planning with stochastic rewards (ablation) | Online Backward-Dyna · discrete navigation task, stochastic rewards | speed of finding optimal policy | Improved |
| extremely stochastic rewards with backward models (ablation) | Online Backward-Dyna · discrete navigation task, extreme reward stochasticity | model accuracy | Hurt |
| model misspecification applied to forward vs backward planning (ablation) | forward planning · discrete navigation task | performance | Hurt |
Closest papers by meaning
- Hindsight Credit Assignment (NeurIPS 2019 · similarity 0.92)
- Hindsight PRIORs for Reward Learning from Human Preferences (ICLR 2024 · similarity 0.87)
- Prospective Hindsight: Self-Calibrating Reinforcement Learning via Prediction–Reality Gaps (NeurIPS 2026 · similarity 0.87)
- Beyond Prediction: Steering VLM Agents with Retrospective World Modeling (NeurIPS 2026 · similarity 0.86)
- Reward-Respecting Subtasks for Model-Based Reinforcement Learning (Abstract Reprint) (AAAI 2024 · similarity 0.86)
- On Shallow Planning Under Partial Observability (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
