Research map / Reinforcement learning / RL from feedback
What Is It You Really Want of Me? Generalized Reward Learning with Biased Beliefs about Domain Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| learn human's biased belief about dynamics (GeReL) vs MaxEnt-IRL | reward learning · Reversed Up & Down | d(R): 23.32 -> 12.17 | Helped |
| learn human's biased belief about dynamics (GeReL) vs MaxEnt-IRL | reward learning · Rotated Belief | d(R): 28.02 -> 12.61 | Helped |
| GeReL vs SERD (soft value iteration) | reward and dynamics learning · Reversed Up & Down | d(R): 16.43 -> 12.17 | Helped |
| GeReL vs SERD | reward and dynamics learning · Rotated Belief | d(R): 17.62 -> 12.61 | Helped |
| update domain dynamics belief (GeReL) vs fixed true dynamics (GeReL−) (ablation) | GeReL · Reversed Up & Down | d(Θ): 0.56 -> 0.06 | Helped |
| update domain dynamics belief vs GeReL− (ablation) | GeReL · Rotated Belief | d(Θ): 0.91 -> 0.08 | Helped |
| update domain dynamics belief vs GeReL− (ablation) | GeReL · Reversed Up & Down | d(π): 1.46 -> 0.11 | Helped |
| update domain dynamics belief vs GeReL− (ablation) | GeReL · Rotated Belief | d(π): 1.43 -> 0.23 | Helped |
| GeReL vs SERD on belief estimation | reward and dynamics learning · Reversed Up & Down | d(Θ): 0.38 -> 0.06 | Helped |
| GeReL vs SERD on policy KL | reward and dynamics learning · Rotated Belief | d(π): 0.63 -> 0.23 | Helped |
| robot appearance (humanoid vs mobile) biases belief about spilling coffee under umbrella | human participants · Coffee Robot user study | Q1 Likert: 2.92 -> 3.58 | Helped |
| robot appearance (humanoid vs mobile) changes concern about robot getting wet | human participants · Coffee Robot user study | Q2 Likert: 2.83 -> 3.67 | Helped |
Closest papers by meaning
- Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning (NeurIPS 2024 · similarity 0.88)
- Information-Theoretic Reward Decomposition for Generalizable RLHF (NeurIPS 2025 · similarity 0.87)
- Capturing Individual Human Preferences with Reward Features (NeurIPS 2025 · similarity 0.86)
- Hindsight PRIORs for Reward Learning from Human Preferences (ICLR 2024 · similarity 0.86)
- DUO: Diverse, Uncertain, On-Policy Query Generation and Selection for Reinforcement Learning from Human Feedback (AAAI 2025 · similarity 0.86)
- Post-hoc Reward Calibration: A Case Study on Length Bias (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.