Research map / Large language models / Alignment and preferences
TUR-DPO: Topology- and Uncertainty-Aware Direct Preference Optimization
Abdulhady abas, Fatemeh Daneshfar, Seyedali Mirjalili, Mourad Oussalah
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add topology and uncertainty signals to DPO | DPO · GSM8K | EM (%): 58.7 -> 62.8 | Improved |
| use TUR-DPO instead of ORPO | preference optimization · GSM8K | EM (%): 59.4 -> 62.8 | Improved |
| use TUR-DPO instead of SimPO | preference optimization · GSM8K | EM (%): 60.1 -> 62.8 | Improved |
| use TUR-DPO instead of PPO RLHF | preference optimization · HH single-turn (human eval) | win-rate (%): 67.9 -> 67.2 | No clear effect |
| increase re-elicited graphs K from 1 to 3 (ablation) | TUR-DPO · GSM8K | EM (%): 61.7 -> 62.8 | Improved |
| increase re-elicited graphs K from 3 to 4 (ablation) | TUR-DPO · GSM8K | EM (%): 62.8 -> 62.9 | No clear effect |
| use a stronger topology extractor (ablation) | TUR-DPO · GSM8K | win-rate (%): 62.8 -> 64.1 | Improved |
| use a weaker topology extractor (ablation) | TUR-DPO · GSM8K | win-rate (%): 62.8 -> 59.8 | Hurt |
| add uncertainty-weighted updates under 20% label flips (ablation) | DPO · GSM8K | win-rate (%): 47.6 -> 55.9 | Improved |
| add post-processing format validation (ablation) | TUR-DPO · GSM8K | EM (%): 62.8 -> 65.3 | Improved |
| add TUR-DPO training to DPO | DPO · GSM8K | ECE: 0.101 -> 0.087 | Improved |
| add TUR-DPO training to DPO | DPO · GSM8K | formatting/missing-answer errors: 23 -> 41 | Hurt |
Closest papers by meaning
- 3D-Properties: Identifying Challenges in DPO and Charting a Path Forward (ICLR 2025 · similarity 0.91)
- Language Models as Implicit Tree Search (ICML 2025 · similarity 0.90)
- AlphaDPO: Adaptive Reward Margin for Direct Preference Optimization (ICML 2025 · similarity 0.90)
- Earlier Tokens Contribute More: Learning Direct Preference Optimization From Temporal Decay Perspective (ICLR 2025 · similarity 0.90)
- Explicit Preference Optimization: No Need for an Implicit Reward Model (ICML 2025 · similarity 0.89)
- TPO: Aligning Large Language Models with Multi-branch & Multi-step Preference Trees (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
