AI Paper Atlas

Research map / Large language models / Alignment and preferences

SafeDPO: A Simple Approach to Direct Preference Optimization with Enhanced Safety

Geon-Hyeong Kim, Yu Jin Kim, Byoungjip Kim, Honglak Lee, Kyunghoon Bae, Youngsoo Jang, Moontae Lee

ICLR 2026 · Large language models › Alignment and preferences · approach: Active and token-level preference learning · 32 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
apply safety-aware pair transformation (swap/discard pairs) instead of plain DPO on helpfulness preferences (ablation)DPO · PKU-SafeRLHF-30K test (model-based)harmless ratio (%): 38.6 -> 97.24Improved
train SafeDPO instead of only filtering pairs with unsafe preferred responses (ablation)DPO-SAFEBETTER · PKU-SafeRLHF-30K testharmless ratioImproved
add safety margin Delta (0 to 10) (ablation)SafeDPO · PKU-SafeRLHF-30K testharmless ratioNo clear effect
increase safety margin to Delta=50 (ablation)SafeDPO · PKU-SafeRLHF-30K testhelpfulnessHurt
add safety margin Delta=10 to standard DPO helpfulness training (ablation)DPO-HELPFUL · PKU-SafeRLHF-30K test (model-based)harmless ratio (%): 38.6 -> 51.63Improved
add safety margin Delta=10 to DPO-HARMLESS (ablation)DPO-HARMLESS · PKU-SafeRLHF-30K test (model-based)harmless ratio (%): 76.82 -> 93.88Improved
add safety margin Delta=10 to DPO-SAFEBETTER (ablation)DPO-SAFEBETTER · PKU-SafeRLHF-30K test (model-based)harmless ratio (%): 50.5 -> 62.13Improved
scale reference model from 1.5B to 13B (ablation)SafeDPO · PKU-SafeRLHF-30K test (model-based)harmless ratio (%): 96.38 -> 97.00No clear effect
use SafeDPO instead of SafeRLHFSafeRLHF · Human evaluationsafety: 0.932 -> 0.943No clear effect
apply SafeDPO instead of SFTSFT · Human evaluationhelpfulness: 0.868 -> 0.499Hurt
use SafeDPO hard-constraint training on benign-lookalike promptsSafeDPO · XSTestover-refusal (%): 0 -> 12.4 (DPO-HELPFUL vs SafeDPO)Hurt
use SafeDPO instead of DPO-HELPFULDPO-HELPFUL · XSTestharmless ratio (%): 14.5 -> 100Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy