AI Paper Atlas

Research map / Reinforcement learning / RL from feedback

PAWS: Preference Learning with Advantage-Weighted Segments

Aleksandar Taranovic, Onur Celik, Niklas Freymuth, Ge Li, Serge Thilges, Huy Le, Tai Hoang, Rania Rayyes, Gerhard Neumann

ICML 2026 · Reinforcement learning › RL from feedback · approach: Preference-based RL reward learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
update policy on segment-level advantages instead of single state-action advantages (ablation)PAWS (MLP) · Meta-World 10 tasks, 500 preferencessuccess rate (%): 67 -> 78Improved
update policy on segment-level advantages instead of single state-action advantages (ablation)PAWS (Trans.) · Meta-World 10 tasks, 50 preferencessuccess rate (%): 44 -> 52Improved
increase segment length under single-step updates (ablation)state-action-based update · Meta-World 10 taskssuccess rateHurt
shorten update-time segments from 64 to 8 with fixed advantage model (ablation)PAWS (Trans.) · Meta-World 10 tasks, 500 preferencessuccess rate (%): 78.2 -> 64.3Hurt
lower effective sample size n_eff to 10% with 500 preferences (ablation)PAWS · Peg Insert Side, Sweep Into, Drawer Opensuccess rateImproved
use larger n_eff with only 50 preferences (ablation)PAWS · Peg Insert Side, Sweep Into, Drawer Opensuccess rateImproved
use segment-based instead of step-based update for rank preservation (ablation)PAWS · Spearman rank correlation with expertSpearman rs: 0.05 -> 0.22Improved
use Transformer instead of MLP advantage function (ablation)PAWS · Meta-World 10 taskssuccess rate (%): 78.3 -> 78.2 (500 prefs)No clear effect
add KL regularization to CPLCPL · Meta-World 10 taskssuccess rate (%): 67.6 -> 67.3 (500 prefs)No clear effect
use PAWS with human-collected preferencesPAWS (MLP) · Meta-World Door Open, 50 preferencessuccess rate (%): 73.6 -> 86.0 (BC -> PAWS MLP)Improved
use baselines with only 50 preferences vs behavior cloningCPL, CPL+KL, IPL · Meta-World 10 tasks, 50 preferencessuccess rate (%): 46.2 -> 42.9 (CPL)Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy