AI Paper Atlas

Research map / Large language models / Alignment and preferences

Alignment-Aware Decoding

Frédéric Berdoz, Luca A Lanzendörfer, René Caky, Roger Wattenhofer

ICML 2026 · Large language models › Alignment and preferences · approach: Robust and calibrated DPO

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
decode with DPO/SFT log-ratio token advantage (AAD) instead of greedy DPOLlama 3B DPO · Skyworkaverage oracle reward: 8.45 -> 13.71Improved
AAD instead of EFT and Bo2 at similar computeLlama 3B DPO · Argillaaverage oracle reward: EFT 4.54, Bo2 3.02 -> AAD 5.64Improved
replace SFT reference with external amateur Llama Instruct models (contrastive decoding) (ablation)AAD on Llama 8B DPO · Skyworkaverage oracle reward: 19.27 -> 13.08Hurt
set min-alpha token filter around 0.1 to 0.2 (ablation)AAD · Skywork, Nectar, Argilla (Llama 3B LoRA)oracle rewardMixed
use full fine-tuning instead of LoRA for the DPO model (ablation)AAD · six datasets, Llama 1B/3Boracle reward: HHRLHF 1B: EFT 0.26 vs AAD -0.06Mixed
AAD instead of Bo2 as training data grows (ablation)AAD on Llama 3B full FT · Skyworkoracle reward: win rate 0.511 -> 0.746 (1% -> 100% data)Improved
iterative DPO using AAD-generated chosen samples, trained from SFT (Iter 1') (ablation)Llama DPO · UltraFeedback 10%R score: R 2.21 for AAD decodingImproved
second iteration of iterative DPO (ablation)Llama DPO · Argilla 5%R score (AAD decoding): 2.25 -> -2.13Hurt
entropy thresholding in beam search on AAD score (ablation)AAD beam search · Skywork alpha 0.7scoreImproved
use standard beam search with more beams (ablation)beam search · Argilla, Skywork, NectarscoreNo clear effect
train DPO for more epochs (ablation)AAD · Skywork Llama 3Boracle rewardHurt
raise DPO beta (ablation)AAD vs Bo2 and greedy · Argillarelative loss of oracle scoreMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy