AI Paper Atlas

Research map / Large language models / Alignment and preferences

Bootstrapping LLMs via Preference-Based Policy Optimization

Chen Jia

AAAI 2026 · Large language models › Alignment and preferences · approach: Robust and calibrated DPO · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
PbPO online bootstrapping (300 episodes) instead of offline DPOLLaMA2-7B · 5-benchmark averageaccuracy: 49.7 -> 57.1 (seq RM)Improved
PbPO instead of Online PPOQwen2-7B · GSM8Kaccuracy: 79.2 -> 85.2Improved
more online PbPO episodes (100 -> 300) (ablation)PbPO seq RM, LLaMA2-7B · 5-benchmark averageaccuracy: 52.7 -> 57.1Improved
token-level RM instead of sequence-level RM (ablation)PbPO · GSM8K (LLaMA2-7B)accuracy: 58.1 -> 60.2Improved
token-level RM instead of sequence-level RM (ablation)PbPO · AGIEval/MMLU (LLaMA2-7B)accuracy: AGIEval 43.7 -> 43.5; MMLU 57.1 -> 56.3Mixed
remove reward-agnostic exploration (no enhancer policy) (ablation)PbPO · AGIEval/ARC-C/MMLU/GSM8KaccuracyHurt
remove reward-aware exploration (pipeline RM then policy) (ablation)PbPO · AGIEval/ARC-C/MMLU/GSM8KaccuracyHurt
set conservatism beta > 0 instead of 0 (ablation)PbPO · LLaMA2-7BaccuracyImproved
larger reward model (3B up to 70B) (ablation)PbPO · LLaMA2-7BaccuracyImproved
online iterative methods (Online DPO, Online PPO, Best-of-N) vs offline RLHFLLaMA2-7B · 5-benchmark averageaccuracy: DPO 49.7 -> Online DPO 50.8No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy