AI Paper Atlas

Research map / Large language models / Alignment and preferences

Preference Robustness for DPO with Applications to Public Health

Cheol Woo Kim, Shresth Verma, Mauricio Tec, Milind Tambe

AAAI 2026 · Large language models › Alignment and preferences · approach: DPO design choices · 5 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use DPO-PRO (rho=0.03) instead of DPO under high label-flip noiseDPO · UltraFeedback, high noise (alpha=0.6)win rate (%): 26.97 -> 28.98Improved
use DPO-PRO (rho=0.008) instead of DPO with no noiseDPO · UltraFeedback, no noisewin rate (%): 43.77 -> 43.24Hurt
use DPO-PRO (rho=0.03) instead of DrDPO under low noiseDrDPO · UltraFeedback, low noise (alpha=0.3)win rate (%): 38.84 -> 33.33Hurt
use DPO-PRO (rho=0.03) instead of DPO on evaluation reward under high noiseDPO · UltraFeedback, high noise (alpha=0.6)evaluation reward: 400.44 -> 499.59Improved
use DPO-PRO (rho=0.1) instead of DPO on evaluation reward with no noiseDPO · UltraFeedback, no noiseevaluation reward: 877.01 -> 781.28Hurt
use DPO-PRO (rho=0.008) instead of DPO on evaluation reward under low noiseDPO · UltraFeedback, low noise (alpha=0.3)evaluation reward: 729.12 -> 713.00Hurt
increase robustness radius rho (ablation)DPO-PRO · UltraFeedback, win ratewin rate (%): rho 0.008: 43.24 -> 28.75; rho 0.03: 41.71 -> 28.98Mixed
use DPO-PRO (rho=0.3) instead of DPO on ground-truth reward, low noiseDPO · ARMMAN RMAB, low noiseavg ground-truth reward: 5707.09 -> 5904.7Improved
use DPO-PRO (rho=0.3) instead of DPO on ground-truth reward, no noiseDPO · ARMMAN RMAB, no noiseavg ground-truth reward: 5465.36 -> 5797.1Improved
use DPO-PRO (rho=0.03) instead of DPO on win rate, low noiseDPO · ARMMAN RMAB, low noisewin rate (%): 13.84 -> 13.5No clear effect
use DPO-PRO (rho=0.3) instead of DPO on win rate, low noiseDPO · ARMMAN RMAB, low noisewin rate (%): 13.84 -> 17.8Improved
use DPO-PRO (rho=0.1) instead of DPO on win rate, no noiseDPO · ARMMAN RMAB, no noisewin rate (%): 15.38 -> 17.7Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy