AI Paper Atlas

Research map / Large language models / Alignment and preferences

Sharpness-Aware Minimization in Logit Space Efficiently Enhances Direct Preference Optimization

Haocheng Luo, Zehang Deng, Thanh-Toan Do, Mehrtash Harandi, Dinh Phung, Trung Le

ICLR 2026 · Large language models › Alignment and preferences · approach: DPO design choices · 1 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add logits-SAM (perturb only output layer)DPO · HH vs chosenwin rate (%): 56.35 -> 60.51Improved
add logits-SAMDPO · TL;DR vs SFTwin rate (%): 84.21 -> 89.58Improved
add logits-SAMSLiC-HF · HH vs SFTwin rate (%): 65.27 -> 71.87Improved
add logits-SAMCPO · TL;DR vs chosenwin rate (%): 39.38 -> 45.41Improved
add logits-SAMDPO (Mistral-7B) · Arena-Hard v0.1win rate (%): 19.0 -> 23.1Improved
add logits-SAMCPO (Mistral-7B) · AlpacaEval 2 LCLC win rate (%): 8.97 -> 13.32Improved
add logits-SAMDPO (Mistral-7B) · MT-Benchscore: 5.49 -> 5.79Improved
add logits-SAMDPO (Gemma-2B-IT) · SorryBench test refusalrefusal rate: 0.7077 -> 0.7538Improved
add logits-SAMCHES (Gemma-2B-IT) · SorryBench test refusalrefusal rate: 0.7846 -> 0.8769Improved
increase rho from 1e-4 to 1e-2 (ablation)logits-SAM DPO · HH vs SFTwin rate (%): 72.28 -> 65.49Hurt
use rho = 1e-5 instead of 0 (AdamW) (ablation)logits-SAM DPO · HH vs SFTwin rate (%): 70.52 -> 69.47 (vs chosen 56.35 -> 58.27)Mixed
use logits-SAM instead of vanilla SAM (ablation)DPO · Pythia-2.8B training costwall-clock time: adds ~2-3% time; vanilla SAM nearly doubles step time and OOMsImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy