Research map / Large language models / Alignment and preferences
Task-Aware Preference Calibration for Direct Preference Optimization
Mengyang Li, Zhong Zhang, Pinlong Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add task conditioning to meta-learned calibration (Uniform-TAPC to TAPC) (ablation) | TAPC · UltraFeedback Creative prompts, 20% noise | win rate (%): 88.5 -> 92.5 | Improved |
| add task conditioning to meta-learned calibration (ablation) | TAPC · UltraFeedback Factual prompts, 20% noise | win rate (%): 93.5 -> 94.2 | No clear effect |
| add task conditioning to meta-learned calibration (ablation) | TAPC · UltraFeedback Open prompts, 20% noise | win rate (%): 90.2 -> 92.8 | Improved |
| add a task encoder over no task encoding (ablation) | TAPC · UltraFeedback, 20% noise | win rate (%): 88.5 -> 90.2 | Improved |
| use mean pooling instead of first-token hidden state for the task encoder (ablation) | TAPC task encoder · UltraFeedback, 20% noise | win rate (%): 89.2 -> 90.2 | Improved |
| use mean pooling instead of last-token hidden state (ablation) | TAPC task encoder · UltraFeedback, 20% noise | win rate (%): 89.0 -> 90.2 | Improved |
| replace learned embedding with oracle task labels (ablation) | TAPC task encoder · UltraFeedback, 20% noise | win rate (%): 90.2 -> 90.0 | No clear effect |
| use meta-learning instead of joint training without reference set (ablation) | TAPC · UltraFeedback, 20% noise | win rate (%): 88.0 -> 90.2 | Improved |
| use meta-learning instead of joint training without reference set (ablation) | TAPC · UltraFeedback, 40% noise | win rate (%): 78.5 -> 84.5 | Improved |
| use meta-learning instead of joint training with reference regularization (ablation) | TAPC · UltraFeedback, 40% noise | win rate (%): 80.2 -> 84.5 | Improved |
| increase reference set size from 100 to 200 (ablation) | TAPC · UltraFeedback, 20% noise | win rate (%): 88.5 -> 90.2 | Improved |
| increase reference set size from 200 to 300 (ablation) | TAPC · UltraFeedback, 20% noise | win rate (%): 90.2 -> 90.5 | No clear effect |
Closest papers by meaning
- Reward-Augmented Data Enhances Direct Preference Alignment of LLMs (ICML 2025 · similarity 0.88)
- Sequential Preference Optimization: Multi-Dimensional Preference Alignment with Implicit Reward Modeling (AAAI 2025 · similarity 0.88)
- Regularized Conditional Diffusion Model for Multi-Task Preference Alignment (NeurIPS 2024 · similarity 0.87)
- Improving Context-Aware Preference Modeling for Language Models (NeurIPS 2024 · similarity 0.87)
- Pairwise Calibrated Rewards for Pluralistic Alignment (NeurIPS 2025 · similarity 0.87)
- Preference Learning for AI Alignment: a Causal Perspective (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
