Research map / Reinforcement learning / Offline RL
Offline Reinforcement Learning for Mixture-of-Expert Dialogue Management
Dhawal Gupta, Yinlam Chow, Azamat Tulepbergenov, Mohammad Ghavamzadeh, Craig Boutilier
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use model-based evaluation with a learned next-user model rather than model-free Q scoring | offline RL dialogue managers · Reddit Casual and Cornell | return | Improved |
| Use MoE-specific SAIQL with all expert candidate utterances rather than the best standard offline RL method | MoE-LM dialogue management · Reddit Casual, model-based | return: 4.25 ± 0.12 -> 4.65 ± 0.06 | Improved |
| Use MoE-specific FtLE rather than the best standard offline RL method | MoE-LM dialogue management · Cornell, model-free | return: -1.32 ± 0.19 -> -0.39 ± 0.24 | Improved |
| Use MoE-specific MoE-VRL rather than the best standard offline RL method | MoE-LM dialogue management · Cornell, model-based | return: 1.47 ± 0.15 -> 3.62 ± 0.17 | Improved |
| Use MoE-specific methods that incorporate all experts rather than the best standard offline RL methods | MoE-LM dialogue management · Reddit Casual and Cornell | return | Mixed |
| Use FtLE or MoE-VRL, which model expert-specific long-term value, instead of SAIQL | MoE-LM dialogue management · Cornell, model-free and model-based | return | Improved |
| Use IQL rather than SAC, EnsQ, or KLC among standard offline RL methods | MoE primitive LM dialogue management · most model-based and model-free evaluations | return | Mixed |
| Use IQL rather than BC | offline dialogue management · Reddit Casual and Cornell | return | Improved |
| Use standard offline RL rather than per-step reward-maximizing Bandits | standard offline RL dialogue managers · Reddit Casual and Cornell | return | No clear effect |
| Use MoE-aware offline RL rather than per-step Bandit greedification | MoE-LM dialogue management · Reddit Casual and Cornell | return | Improved |
| Use SAC rather than EnsQ or KLC | standard offline RL dialogue management · Reddit Casual, model-free | return | Mixed |
| Apply dropout-based ensemble Q or KL regularization in the fixed MoE latent space | EnsQ and KLC · open-domain dialogue management | return | Hurt |
Closest papers by meaning
- LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models (ICML 2025 · similarity 0.90)
- ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL (ICML 2024 · similarity 0.89)
- Planning without Search: Refining Frontier LLMs with Offline Goal-Conditioned RL (NeurIPS 2025 · similarity 0.89)
- Plug-and-Play Policy Planner for Large Language Model Powered Dialogue Agents (ICLR 2024 · similarity 0.88)
- Offline RL by Reward-Weighted Fine-Tuning for Conversation Optimization (NeurIPS 2025 · similarity 0.88)
- Learning to Clarify: Multi-turn Conversations with Action-Based Contrastive Self-Training (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
