Research map / Reinforcement learning / RL from feedback
Sequential Preference Ranking for Efficient Reinforcement Learning from Human Feedback
Minyoung Hwang, Gunmin Lee, Hogun Kee, Chan Woo Kim, Kyungjae Lee, Songhwai Oh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use root pairwise comparison instead of pairwise (ablation) | MRN · DMControl locomotion (avg) | reward after convergence: 29.0% | Improved |
| use sequential pairwise comparison instead of pairwise (ablation) | MRN · DMControl locomotion (avg) | reward after convergence: 10.3% | Improved |
| use root pairwise comparison instead of pairwise (ablation) | MRN · Meta-World manipulation (avg) | success rate: 25.0% | Improved |
| use sequential pairwise comparison instead of pairwise (ablation) | MRN · Meta-World manipulation (avg) | success rate: 1.2% | No clear effect |
| use root pairwise comparison instead of sequential pairwise (ablation) | MRN · Meta-World Door Open | success rate: 77.4 ± 40.8 -> 97.8 ± 5.0 | Improved |
| use sequential pairwise instead of pairwise (ablation) | MRN · Meta-World Sweep Into | success rate: 68.4 ± 35.2 -> 55.7 ± 48.1 | Hurt |
| use sequential pairwise instead of pairwise (ablation) | MRN · Meta-World Hammer | success rate: 31.0 ± 24.7 -> 30.8 ± 35.4 | No clear effect |
| use root pairwise instead of pairwise (ablation) | MRN · DMControl Quadruped Walk | reward after convergence: 353.5 ± 183.9 -> 728.8 ± 274.1 | Improved |
| use root pairwise instead of pairwise (ablation) | MRN · DMControl Hopper Hop | reward after convergence: 26.2 ± 37.0 -> 100.1 ± 70.7 | Improved |
| increase number of feedbacks (0.1K to 4K) (ablation) | sequential and root pairwise · DMControl Quadruped Walk | average reward | Improved |
| use sequential pairwise instead of pairwise, real robot (ablation) | DQN with MRN · Real UR-5 block placing | success rate: 34.4 -> 41.0 | Improved |
| use root pairwise instead of pairwise, real robot (ablation) | DQN with MRN · Real UR-5 block placing | reward model accuracy: 75.8 -> 84.9 | Improved |
Closest papers by meaning
- Towards Theoretical Understanding of Sequential Decision Making with Preference Feedback (ICML 2025 · similarity 0.91)
- Comparing Comparisons: Informative and Easy Human Feedback with Distinguishability Queries (ICML 2025 · similarity 0.91)
- Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learning (NeurIPS 2024 · similarity 0.90)
- DUO: Diverse, Uncertain, On-Policy Query Generation and Selection for Reinforcement Learning from Human Feedback (AAAI 2025 · similarity 0.90)
- Adaptive Preference Scaling for Reinforcement Learning with Human Feedback (NeurIPS 2024 · similarity 0.90)
- Listwise Reward Estimation for Offline Preference-based Reinforcement Learning (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
