Research map / Reinforcement learning / RL from feedback
Reward Model Evaluation via Automatically-Ranked Policy Alignment
Aoran Wang, Lei Ou, Yang Yu, Zongzhang Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| generate ranked policies with DGPI instead of noisy BC perturbations (ablation) | PPAC · DoorKey | PotentialShaped score: 0.91 -> 0.98 | Improved |
| generate ranked policies with DGPI instead of noisy BC perturbations (ablation) | PPAC · HalfCheetah | SecondGoal-Slight score: 0.8346 -> 0.7549 | Mixed |
| generate ranked policies with DGPI instead of AIL checkpoints (ablation) | PPAC · HalfCheetah | SecondGoal-Slight/Intense score: 0.9774 -> 0.7549; 0.9549 -> 0.5203 | Improved |
| generate ranked policies with DGPI instead of AIL checkpoints (ablation) | PPAC · DoorKey | SecondGoal-Intense score: 0.6220 -> 0.4800 | Mixed |
| use listwise policy ranking (PPAC) instead of pairwise trajectory preference (TAC) | PPAC · DoorKey | SecondGoal-Intense score: 1.0000 -> 0.4800 | Improved |
| use PPAC instead of NegativeSTARC for PotentialShaped | PPAC · HalfCheetah | PotentialShaped vs GroundTruth equality: STARC 0.0000 vs -0.6745; PPAC 0.9873 vs 0.9873 | Improved |
Closest papers by meaning
- Data with High and Consistent Preference Difference Are Better for Reward Model (AAAI 2025 · similarity 0.91)
- Tool-Augmented Reward Modeling (ICLR 2024 · similarity 0.89)
- Reward Model Learning vs. Direct Policy Optimization: A Comparative Analysis of Learning from Human Preferences (ICML 2024 · similarity 0.89)
- Query-Policy Misalignment in Preference-Based Reinforcement Learning (ICLR 2024 · similarity 0.89)
- Policy Filtration for RLHF to Mitigate Noise in Reward Models (ICML 2025 · similarity 0.89)
- Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithms (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
