Research map / Reinforcement learning / RL from feedback
B-Pref: Benchmarking Preference-Based Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PEBBLE instead of PrefPPO | Preference-based RL · B-Pref environments | normalized returns | Helped |
| Use a teacher that makes mistakes or gives stochastic preferences instead of an Oracle teacher | PEBBLE and PrefPPO · B-Pref environments | normalized returns | Hurt |
| Provide uniform labels for equally preferable segments instead of only perfect preference labels | Preference-based RL · Quadruped-walk | normalized returns | Helped |
| Skip queries involving similar behaviors instead of relying only on perfect preference labels | Preference-based RL · Quadruped-walk | normalized returns | Helped |
| Select queries using uncertainty (ensemble disagreement or entropy) instead of uniform or coverage-based sampling (ablation) | PEBBLE · Quadruped-walk | normalized returns | Helped |
| Use coverage-based query sampling instead of uniform sampling (ablation) | PEBBLE · Quadruped-walk | normalized returns | No clear effect |
| Use a rule-based increase or decay feedback schedule instead of uniform scheduling (ablation) | PEBBLE · Oracle teacher; tested environments | normalized returns | No clear effect |
| Apply label smoothing to the reward model (ablation) | PEBBLE · Walker-walk with Mistake teacher | normalized returns | No clear effect |
Closest papers by meaning
- Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF (ICLR 2025 · similarity 0.91)
- Policy-labeled Preference Learning: Is Preference Enough for RLHF? (ICML 2025 · similarity 0.91)
- Reward Model Learning vs. Direct Policy Optimization: A Comparative Analysis of Learning from Human Preferences (ICML 2024 · similarity 0.91)
- Provable Reward-Agnostic Preference-Based Reinforcement Learning (ICLR 2024 · similarity 0.90)
- Listwise Reward Estimation for Offline Preference-based Reinforcement Learning (ICML 2024 · similarity 0.90)
- Greedy Sampling Is Provably Efficient For RLHF (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.