Research map / Multi-agent systems and games / LLM agents

DialogXpert: Driving Intelligent and Emotion-Aware Conversations Through Online Value-Based Reinforcement Learning with LLM Priors

AAAI 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DialogXpert, which uses a frozen LLM to propose top-k dialogue actions and an online-trained Q-network over fixed BERT embeddings (plus an emotion tracker and LLM critic rewards) to plan proactive dialogues in negotiation, emotional support, and tutoring.

Problem

Frozen LLM dialogue agents are reactive and myopic, while tree-search planners (MCTS) are too costly for real-time proactive, goal-driven dialogue.

Approach

Use a frozen LLM as a top-k action proposer, then select among the candidates with a lightweight Q-network over fixed BERT state-action embeddings trained with TD learning online, with emotion tracking folded into the state and LLM-critic rewards.

Key results

• On ESConv, DialogXpert (Qwen 2.5 14B) reaches AT 2.31 and SR 0.9876, versus DPDP (GPT-3.5-Turbo) at 3.61 and 0.9000 without MCTS.
• On CraigslistBargain, DialogXpert (Qwen 2.5 14B) achieves SR 0.9746 and SL 0.4389, the best SL among reported methods, vs PPDPP SL 0.3376.
• On ExTES, DialogXpert (Qwen 2.5 14B) reports SR 0.9782 and AT 2.57, versus LDPP's 0.903 and 4.132.

Surprising findings

• Removing RL (w/o RL) lowers success substantially on CB, from 0.9746 to 0.8754, showing the online Q-learning step is central to the gains.
• Human evaluation on ESConv shows only modest preference margins over PPDPP (Comforting 52% vs 45%, Overall 51% vs 41%).

Limitations

• Reward mapping from textual critic feedback to scalar values can be subjective (e.g., CIMA partial-credit rewards).
• CIMA English-Italian is high-resource and may not be a challenging tutoring testbed.
• Dynamic state-action space prevents caching, adding computational overhead.

Experimental setup

Datasets: ESConv CIMA CraigslistBargain P4G ExTES

Benchmarks: CraigslistBargain ESConv CIMA P4G ExTES

Metrics: average turns success rate sale-to-list ratio human win/tie/loss preference

Baselines: DialoGPT Standard AnE Proactive ProCoT ICL-AIF PPDPP DPDP UDP LDPP GDP-Zero TRIP

Builds on: PPDPP DPDP LDPP UDP

Closest papers by meaning

• Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents (ICLR 2024 · similarity 0.89)
• RLVER: Reinforcement Learning with Verifiable Emotion Rewards for Empathetic Agents (ICLR 2026 · similarity 0.89)
• Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training (ICLR 2025 · similarity 0.88)
• Simulation-Free Hierarchical Latent Policy Planning for Proactive Dialogues (AAAI 2025 · similarity 0.88)
• Decoupling Conversational Dynamics in Full-Duplex Spoken Models through Reinforcement Learning (NeurIPS 2026 · similarity 0.87)
• Enhancing Personalized Multi-Turn Dialogue with Curiosity Reward (NeurIPS 2025 · similarity 0.87)