Research map / Large language models / Language, safety and interpretability
GENTEEL-NEGOTIATOR: LLM-Enhanced Mixture-of-Expert-Based Reinforcement Learning Approach for Polite Negotiation Dialogue
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GENTEEL-NEGOTIATOR, an LLaMA-3.1-8B-based mixture-of-experts RL dialogue system (negotiation, politeness, keyterm experts plus a multi-reward function) for polite negotiation, and releases the NEGOCHAT tourism dataset.
Problem
Negotiation dialogue systems rarely model politeness alongside negotiation strategy, coherence, and engagingness, so generated responses can be offensive or unproductive in tourism and e-commerce negotiations.
Approach
Combines an LLM dialogue encoder with heuristic MoE experts (negotiation strategy, politeness, keyterm) whose selection is made by a REINFORCE-trained expert determination policy, optimized with a reward mixing negotiation strategy congruence, politeness, coherence, and engagingness.
Key results
- On NEGOCHAT, GENTEEL-NEGOTIATOR reaches PPL 14.72 and BLEU-2 7.26 vs 16.55 and 6.24 for the strongest baseline LLaMA-3.1-8B-finetune.
- On IND, perplexity drops to 1.14 from 2.63 for LLaMA-3.1-8B-finetune, with BERTScore-f1 0.882 vs 0.871.
- Human evaluation on NEGOCHAT gives outcome fairness 4.78 and bargaining efficacy 4.55, versus 3.01 and 3.15 for LLaMA-3.1-8B-finetune.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GENTEEL-NEGOTIATOR instead of LLaMA-3.1-8B-finetune | LLaMA-3.1-8B-Instruct · NEGOCHAT | B-2: 6.24 -> 7.26 | Helped |
| Use GENTEEL-NEGOTIATOR instead of LLaMA-3.1-8B-finetune | LLaMA-3.1-8B-Instruct · NEGOCHAT | ScNS: 0.698 -> 0.751 | Helped |
| Use GENTEEL-NEGOTIATOR instead of LLaMA-3.1-8B-finetune | LLaMA-3.1-8B-Instruct · NEGOCHAT | SdP: 0.682 -> 0.712 | Helped |
| Use GENTEEL-NEGOTIATOR instead of LLaMA-3.1-8B-finetune | LLaMA-3.1-8B-Instruct · NEGOCHAT | ScDC: 0.743 -> 0.781 | Helped |
| Use GENTEEL-NEGOTIATOR instead of LLaMA-3.1-8B-finetune | LLaMA-3.1-8B-Instruct · IND | B-2: 7.24 -> 8.67 | Helped |
| Use GENTEEL-NEGOTIATOR instead of LLaMA-3.1-8B-finetune | LLaMA-3.1-8B-Instruct · IND | ScNS: 0.690 -> 0.842 | Helped |
| Use GENTEEL-NEGOTIATOR instead of LLaMA-3.1-8B-finetune | LLaMA-3.1-8B-Instruct · IND | SdP: 0.727 -> 0.872 | Helped |
| Use GENTEEL-NEGOTIATOR instead of LLaMA-3.1-8B-finetune | LLaMA-3.1-8B-Instruct · IND | ScDC: 0.632 -> 0.789 | Helped |
| Use GENTEEL-NEGOTIATOR instead of LLaMA-3.1-8B-finetune | LLaMA-3.1-8B-Instruct · NEGOCHAT | SC: 3.54 -> 4.17 | Helped |
| Use GENTEEL-NEGOTIATOR instead of LLaMA-3.1-8B-finetune | LLaMA-3.1-8B-Instruct · NEGOCHAT | NC: 3.07 -> 4.31 | Helped |
| Use GENTEEL-NEGOTIATOR instead of LLaMA-3.1-8B-finetune | LLaMA-3.1-8B-Instruct · NEGOCHAT | BE: 3.15 -> 4.55 | Helped |
Surprising findings
- INA, a non-LLM baseline, reaches a notably higher IND engagingness and politeness than some LLM baselines, while ProCoT (GPT-3.5) scores poorly on negotiation and politeness despite strong coherence.
Limitations
- Inter-annotator agreement for negotiation strategy labels is only moderate to fair (kappa 0.3 to 0.7).
- NEGOCHAT is LLM-generated from a small set of 50 human-written seed dialogues, which may limit realism.
- Politeness and coherence rewards rely on classifiers with roughly 70% to 84% accuracy.
Experimental setup
Datasets: NEGOCHAT IND
Benchmarks: NEGOCHAT IND
Metrics: PPL BLEU-2 Distinct-2 BERTScore-f1 response length negotiation strategy congruence politeness score dialogue coherence score engagingness sociopsychological closeness
Baselines: DialoGPT ARDM PersRFI GPT-Critic INA ProCoT (GPT-3.5) LLaMA-3.1-8B-finetune
Builds on: REINFORCE INA GPT-Critic PersRFI Danescu-Niculescu-Mizil politeness classifier
Closest papers by meaning
- How do Large Language Models Navigate Conflicts between Honesty and Helpfulness? (ICML 2024 · similarity 0.85)
- PieArena: Ranking and Profiling Language Agents in Realistic Negotiation Scenarios (NeurIPS 2026 · similarity 0.84)
- A Contextual Combinatorial Bandit Approach to Negotiation (ICML 2024 · similarity 0.84)
- How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis (ICML 2024 · similarity 0.84)
- DialogXpert: Driving Intelligent and Emotion-Aware Conversations Through Online Value-Based Reinforcement Learning with LLM Priors (AAAI 2026 · similarity 0.84)
- Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.