Research map / Reinforcement learning / RL from feedback
Inverse Preference Learning: Preference-based RL without a Reward Function
Joey Hejna, Dorsa Sadigh
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove regularization (lambda = 0) (ablation) | IPL · hop-m-r | normalized score: 73.57 -> 10.41 | Hurt |
| remove regularization (lambda = 0) (ablation) | IPL · walk-m-r | normalized score: 59.92 -> 4.85 | Hurt |
| remove regularization (lambda = 0) (ablation) | IPL · can-mh | normalized score: 57.6 -> 13.8 | Hurt |
| remove data augmentation (ablation) | IPL · hop-m-r | normalized score: 73.57 -> 70.46 | Hurt |
| remove data augmentation (ablation) | IPL · hop-m-e | normalized score: 74.52 -> 51.26 | Hurt |
| remove data augmentation (ablation) | IPL · walk-m-e | normalized score: 108.51 -> 108.91 | No clear effect |
| remove data augmentation (ablation) | IPL · can-ph | normalized score: 74.8 -> 68.6 | Hurt |
| remove data augmentation (ablation) | IPL · lift-ph | normalized score: 97.60 -> 98.0 | No clear effect |
| replace IQL with XQL (ablation) | IPL · hop-m-r | normalized score: 73.57 -> 80.4 | Improved |
| replace IQL with XQL (ablation) | IPL · walk-m-e | normalized score: 108.51 -> 75.16 | Hurt |
| replace IQL with XQL (ablation) | IPL · hop-m-e | normalized score: 74.52 -> 54.3 | Hurt |
| use IPL instead of explicit Markovian reward model (MR) | IQL · MetaWorld Button Press, 4000 queries | success rate: 78.3 -> 90.2 | Improved |
Closest papers by meaning
- O-MAPL: Offline Multi-agent Preference Learning (ICML 2025 · similarity 0.92)
- Policy-labeled Preference Learning: Is Preference Enough for RLHF? (ICML 2025 · similarity 0.92)
- Provable Reward-Agnostic Preference-Based Reinforcement Learning (ICLR 2024 · similarity 0.91)
- Reward Model Learning vs. Direct Policy Optimization: A Comparative Analysis of Learning from Human Preferences (ICML 2024 · similarity 0.91)
- Preference Elicitation for Offline Reinforcement Learning (ICLR 2025 · similarity 0.90)
- Decoding Global Preferences: Temporal and Cooperative Dependency Modeling in Multi-Agent Preference-Based Reinforcement Learning (AAAI 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
