Research map / Reinforcement learning / RL from feedback
Policy Likelihood-based Query Sampling and Critic-Exploited Reset for Efficient Preference-based Reinforcement Learning
Jongkook Heo, Jaehoon Kim, Young Jae Lee, Min Gu Kwak, Youngjoon Park, Seoung Bum Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reset reward estimator at every feedback session | PEBBLE · Walker Walk (100/10) | episode return and reward overestimation: plateau at approximately 150% without reset | Improved |
| add policy likelihood-based query sampling (PLS) (ablation) | PEBBLE · Walker Walk, Door Open, Drawer Open | episode return / success rate | Improved |
| add critic-exploited reset (CER) (ablation) | PEBBLE · Walker Walk, Door Open, Drawer Open | episode return / success rate | Improved |
| combine PLS and CER (ablation) | PEBBLE · Drawer Open (5000/50) | success rate: approximately 100% (CER alone) | Hurt |
| add temporal data augmentation (TA) (ablation) | PoLiCER · Door Open, Drawer Open | score | Mixed |
| add CER on top of PLS (ablation) | PLS-only PoLiCER · critic output | critic overestimation | Improved |
| replace recency-based sampling (QPA) with PLS | QPA · Meta-World | success rate: over 80% | Improved |
| replace QPA with PoLiCER in pixel-based control | QPA · Cheetah Run and Window Open (pixel) | performance | Improved |
| replace PEBBLE with PoLiCER | PEBBLE · Window Open (pixel) | success rate: approximately 80%, doubling PEBBLE | Improved |
| replace PEBBLE with PoLiCER with human feedback | PEBBLE · Walker Walk and Window Open (human labels) | ground truth reward / success rate | Improved |
Closest papers by meaning
- Query-Policy Misalignment in Preference-Based Reinforcement Learning (ICLR 2024 · similarity 0.93)
- Efficient Preference-Based Reinforcement Learning: Randomized Exploration meets Experimental Design (NeurIPS 2025 · similarity 0.89)
- Preference-based Reinforcement Learning beyond Pairwise Comparisons: Benefits of Multiple Options (NeurIPS 2025 · similarity 0.89)
- DUO: Diverse, Uncertain, On-Policy Query Generation and Selection for Reinforcement Learning from Human Feedback (AAAI 2025 · similarity 0.89)
- Provable Reward-Agnostic Preference-Based Reinforcement Learning (ICLR 2024 · similarity 0.88)
- Greedy Sampling Is Provably Efficient For RLHF (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
