Research map / Reinforcement learning / RL from feedback
Reward Uncertainty for Exploration in Preference-based Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add reward-ensemble uncertainty as an intrinsic exploration bonus | PEBBLE · Meta-World manipulation tasks | sample-efficiency | Helped |
| Use reward uncertainty rather than state-visit novelty for exploration | PEBBLE · Meta-World manipulation tasks | sample-efficiency | Helped |
| Add reward-ensemble uncertainty exploration | PEBBLE · Drawer Open, 5000 queries | success rate: 0.94 ± 0.08 -> 0.99 ± 0.02 | Helped |
| Add reward-ensemble uncertainty exploration | PEBBLE · Sweep Into, 5000 queries | success rate: 0.8 ± 0.08 -> 0.9 ± 0.14 | Helped |
| Add reward-ensemble uncertainty exploration | PEBBLE · Door Unlock, 2500 queries | success rate: 0.64 ± 0.45 -> 0.8 ± 0.4 | Helped |
| Add reward-ensemble uncertainty exploration | PEBBLE · Door Open, 4000 queries | success rate: 1 ± 0 -> 1 ± 0 | No clear effect |
| Add reward-ensemble uncertainty exploration | PEBBLE · Door Close, 500 queries | success rate: 0.8 ± 0.4 -> 1 ± 0 | Helped |
| Add reward-ensemble uncertainty exploration | PEBBLE · Window Close, 500 queries | success rate: 0.86 ± 0.28 -> 0.99 ± 0.02 | Helped |
| Add reward-ensemble uncertainty exploration | PrefPPO · Button Press, 20000 queries | success rate: 0.46 ± 0.20 -> 0.64 ± 0.18 | Helped |
| Vary reward ensemble size among 3, 5, and 7 models (ablation) | RUNE · Meta-World manipulation tasks | sample-efficiency | No clear effect |
| Use fewer reward models in the ensemble (ablation) | RUNE · Meta-World manipulation tasks | sample-efficiency | No clear effect |
| Vary queries per feedback session between 10 and 50 (ablation) | PEBBLE + RUNE · Meta-World manipulation tasks | success rate | No clear effect |
Closest papers by meaning
- Efficient Preference-Based Reinforcement Learning: Randomized Exploration meets Experimental Design (NeurIPS 2025 · similarity 0.93)
- DUO: Diverse, Uncertain, On-Policy Query Generation and Selection for Reinforcement Learning from Human Feedback (AAAI 2025 · similarity 0.90)
- Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning (NeurIPS 2024 · similarity 0.90)
- Towards Theoretical Understanding of Sequential Decision Making with Preference Feedback (ICML 2025 · similarity 0.90)
- Value-Incentivized Preference Optimization: A Unified Approach to Online and Offline RLHF (ICLR 2025 · similarity 0.90)
- Query-Policy Misalignment in Preference-Based Reinforcement Learning (ICLR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.