Research map / Large language models / Alignment and preferences
Simulated Rewards, Skewed Strategies: Tracing the Acquired Preference Bias in LLM-Based Dialogue Planners
Heyan Huang, Yizhe Yang, Huashan Sun, Jiawei Li, Yang Gao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| RL-trained planners (PPDPP, TRIP) vs prompt-only Standard | dialogue planners · Persuasion, GPT-4.1 | preference bias B: 1.02 -> 2.92 (PPDPP) | Hurt |
| RL-trained PPDPP vs Standard prompting | dialogue planner · Persuasion, GPT-4.1 | strategy entropy: 2.08 -> 0.26 | Hurt |
| search-based GDPZero/DPDP vs Standard | dialogue planners · Emotional Support, Qwen-32B | preference bias B: 0.73 -> 0.58 (GDPZero) | Improved |
| GDPZero/DPDP vs Standard in negotiation and persuasion | dialogue planners · Price Negotiation, GPT-4.1 | preference bias B: 1.22 -> 1.52 (GDPZero) | Mixed |
| change the user-simulator LLM (ablation) | dialogue planners · role decomposition, negotiation | convergence score: 0.6-0.8 -> 0.2-0.4 | Hurt |
| add persona diversification to user simulator (ablation) | GDPZero · Emotional Support, GPT-4.1 | preference bias B: 1.20 -> 2.17 | Hurt |
| add persona diversification to user simulator (ablation) | PPDPP · Price Negotiation, GPT-4.1 | preference bias B: 1.64 -> 2.20 | Hurt |
| add persona diversification to user simulator (ablation) | PPDPP · Persuasion, GPT-4.1 | preference bias B: 2.92 -> 2.97 | No clear effect |
| add inappropriate strategies to the action space | PPDPP and one other planner · negotiation, support, persuasion | inappropriate-strategy adoption | Hurt |
Closest papers by meaning
- Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents (ICLR 2024 · similarity 0.88)
- Consistently Simulating Human Personas with Multi-Turn Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Broaden your SCOPE! Efficient Multi-turn Conversation Planning for LLMs with Semantic Space (ICLR 2025 · similarity 0.86)
- LLM Agents Can Be Choice-Supportive Biased Evaluators: An Empirical Study (AAAI 2025 · similarity 0.86)
- FairMT-Bench: Benchmarking Fairness for Multi-turn Dialogue in Conversational LLMs (ICLR 2025 · similarity 0.86)
- Simulation-Free Hierarchical Latent Policy Planning for Proactive Dialogues (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
