AI Paper Atlas

Research map / Large language models / Alignment and preferences

Causally Robust Preference Learning with Reasons

Minjune Hwang, Yigit Korkmaz, Daniel Seita, Erdem Biyik

ICLR 2026 · Large language models › Alignment and preferences · approach: Reward model design and preference models

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add reason-orthogonal consistency loss (ablation)ReCouPLe · ManiSkill 2-task OODmean OOD reward accuracy: 0.726 -> 0.872Improved
add reason-orthogonal consistency loss (ablation)ReCouPLe · ManiSkill 4-task OODmean OOD reward accuracy: 0.745 -> 0.878Improved
add reward-ratio regularizer on top of consistency removal (ablation)ReCouPLe · ManiSkill OODreward accuracy: 0.727 -> 0.726No clear effect
add rationale projection with equality constraint (ReCouPLe-EC)BT-Multi · ManiSkill Pick-Larger 2-task OODreward accuracy: 0.600 -> 0.820Improved
add naive reason auxiliary loss (RFP)BT-Multi · ManiSkill Pick-Larger 2-task OODreward accuracy: 0.600 -> 0.620No clear effect
use inequality constraint (IC) instead of equality (EC)ReCouPLe · ManiSkill Pick-Larger 2-task OODreward accuracy: 0.820 -> 0.633Hurt
use inequality constraint (IC) instead of equality (EC)ReCouPLe · Meta-World Push-Wall visual OODreward accuracy: 0.760 -> 0.900Improved
add ReCouPLe reasons for transfer to novel taskBT-Multi · Meta-World Pick-Place novelreward accuracy: 0.547 -> 0.663Improved
add ReCouPLe-EC to visual confounderBT · Meta-World Push visual OODreward accuracy: 0.233 -> 0.713Improved
use diversified paraphrased rationales (16 per task) (ablation)ReCouPLe-EC · ManiSkill Pick OODreward accuracy: 0.820 -> 0.867Improved
provide rationales for only 25% of pairs (ablation)ReCouPLe-EC · ManiSkill Pick OODreward accuracy: 0.540 -> 0.783Improved
increase preference queries from 1000 to 2000 per task (ablation)ReCouPLe-EC · ManiSkill Pick OODreward accuracy: 0.820 -> 0.913Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy