Research map / Reinforcement learning / Visual and representation RL
Rewriting History with Inverse RL: Hindsight Inference for Policy Improvement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Relabel experience with inverse RL instead of final-state, random, future-state, or no relabeling | HIPI-RL with SAC · Six goal-reaching domains | learning speed | Helped |
| Use soft inverse-RL relabeling rather than the comparison strategies | HIPI-RL with SAC · Sparse 2D reacher and Jaco reaching | learning progress | Helped |
| Relabel data with inverse RL | HIPI-RL with SAC · Discrete 2D reacher goals, walker linear rewards, and Sawyer reach with goal margins | sample efficiency | Helped |
| Relabel data with inverse RL | HIPI-RL with SAC · Discrete 2D reacher goals, walker linear rewards, and Sawyer reach with goal margins | asymptotic reward | Helped |
| Infer task labels with inverse RL before behavior cloning, rather than clone the mixed dataset without task labels | HIPI-BC · Half-cheetah goal velocities, quadruped linear rewards, and discrete manipulation tasks | final reward | Helped |
| Remove partition-function normalization from inverse-RL relabeling (ablation) | HIPI-BC · Half-cheetah with reward bias on the first task | behavior-cloning performance | Hurt |
| Add partition-function normalization to inverse-RL relabeling (ablation) | HIPI-BC · Half-cheetah with reward bias on the first task | robustness to reward scaling | Helped |
| Relabel 100% rather than 50% of replay-buffer samples (ablation) | HIPI-RL with off-policy RL · Off-policy RL experiments | learning performance | Hurt |
| Retain some originally commanded task labels alongside relabeled samples (ablation) | HIPI-RL with off-policy RL · Off-policy RL experiments | learning performance | Helped |
| Use random task relabeling instead of inverse-RL relabeling | Off-policy RL with SAC · Goal-reaching tasks | learning performance | Mixed |
| Use final-state relabeling instead of inverse-RL relabeling | Off-policy RL with SAC · Goal-reaching tasks | learning performance | Mixed |
| Relabel trajectories with the task receiving the largest raw reward, omitting partition-function correction (ablation) | Inverse-RL relabeling · Tasks with different difficulty and reward scales | task-label assignment | Hurt |
Closest papers by meaning
- How Does Goal Relabeling Improve Sample Efficiency? (ICML 2024 · similarity 0.89)
- Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective (ICML 2024 · similarity 0.88)
- Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignment (NeurIPS 2024 · similarity 0.87)
- PIPER: Primitive-Informed Preference-based Hierarchical Reinforcement Learning via Hindsight Relabeling (ICML 2024 · similarity 0.87)
- Efficient Recurrent Off-Policy RL Requires a Context-Encoder-Specific Learning Rate (NeurIPS 2024 · similarity 0.87)
- A Theoretical Framework for Partially-Observed Reward States in RLHF (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.