Research map / Reinforcement learning / RL from feedback
Learning What Matters Now: Dynamic Preference Inference under Contextual Shifts
Xianwei Cao, Dou Quan, Zhenliang Zhang, Shuang Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use preference-conditioned PPO instead of Q-learning (ablation) | DPI · Queue | mean episodic return: 3.74 -> 10.34 | Improved |
| use preference-conditioned PPO instead of Q-learning (ablation) | DPI · Maze | success rate (%): 42.94 -> 59.04 | Improved |
| add KL prior term to the ELBO (ablation) | DPI · Queue post-shift | PS@K | Improved |
| add directional alignment loss (ablation) | DPI · Queue post-shift | PS@K | Improved |
| add self-consistency (stability) regularizer (ablation) | DPI · Queue post-shift | PS@K | Improved |
| use history window H=1 instead of larger windows (ablation) | DPI · Maze | mean episodic return | Hurt |
| increase history window H beyond 9 (ablation) | DPI · Maze | mean episodic return | No clear effect |
| replace ENVELOPE with DPI | ENVELOPE · Queue | success rate (%): 25.10 -> 39.95 | Improved |
| replace ENVELOPE with DPI | ENVELOPE · Maze | mean episodic return: 10.36 -> 30.16 | Improved |
| replace FIXED with DPI | FIXED · Maze | success rate (%): 1.12 -> 59.04 | Improved |
| replace MER-PPO with DPI | MER-PPO · Maze | success rate (%): 0.05 -> 59.04 | Improved |
| replace MER-PPO with DPI | MER-PPO · Maze | mean episodic return: 85.55 -> 30.16 | Hurt |
Closest papers by meaning
- Towards Theoretical Understanding of Sequential Decision Making with Preference Feedback (ICML 2025 · similarity 0.89)
- Bayesian Preference Learning for Test-Time Steerable Reward Models (NeurIPS 2026 · similarity 0.88)
- Preference Learning with Response Time: Robust Losses and Guarantees (NeurIPS 2025 · similarity 0.87)
- AI Alignment with Changing and Influenceable Reward Functions (ICML 2024 · similarity 0.87)
- A Minimaximalist Approach to Reinforcement Learning from Human Feedback (ICML 2024 · similarity 0.87)
- Distributional Preference Learning: Understanding and Accounting for Hidden Context in RLHF (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
