Research map / Reinforcement learning / Policy gradient and actor-critic
BBQ-Networks: Efficient Exploration in Deep Reinforcement Learning for Task-Oriented Dialogue Systems
Zachary Lipton, Xiujun Li, Jianfeng Gao, Lihong Li, Faisal Ahmed, Li Deng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Bayesian Q-network Thompson sampling instead of DQN exploration | Dialogue policy · Full Domain | Success Rate: 0.2693 -> 0.5031 | Improved |
| Use Bayesian Q-network Thompson sampling instead of DQN exploration | Dialogue policy · Domain Extension | Success Rate: 0.3503 -> 0.7120 | Improved |
| Add VIME intrinsic reward to BBQN-MAP (ablation) | BBQN-MAP · Full Domain | Success Rate: 0.5031 -> 0.4856 | Hurt |
| Add VIME intrinsic reward to BBQN-MAP (ablation) | BBQN-MAP · Domain Extension | Success Rate: 0.6852 -> 0.6813 | Mixed |
| Use Monte Carlo sampled target values instead of MAP targets (ablation) | BBQN · Full Domain | Success Rate: 0.5031 -> 0.4877 | Hurt |
| Use Monte Carlo sampled target values instead of MAP targets (ablation) | BBQN · Domain Extension | Success Rate: 0.6852 -> 0.6722 | Mixed |
| Combine VIME intrinsic reward with BBQN-MC (ablation) | BBQN-MC · Domain Extension | Success Rate: 0.6722 -> 0.7120 | Improved |
| Use uniform random exploration in a hybrid with Thompson sampling (ablation) | BBQN · Movie-booking task | performance | No clear effect |
| Pre-fill replay buffer with successful rule-based dialogues (ablation) | BBQN and DQN · Movie-booking task | success | Improved |
| Increase replay-buffer spiking from 100 to 1000 dialogues (ablation) | BBQN-MAP · Movie-booking task | long-term performance | Hurt |
| Train without replay-buffer spiking (ablation) | BBQN and DQN · Movie-booking task | success | Hurt |
Closest papers by meaning
- Efficient Exploration for LLMs (ICML 2024 · similarity 0.85)
- Representation-Based Exploration for Language Models: From Test-Time to Post-Training (ICLR 2026 · similarity 0.85)
- Q-SFT: Q-Learning for Language Models via Supervised Fine-Tuning (ICLR 2025 · similarity 0.85)
- Memory-T1: Reinforcement Learning for Temporal Reasoning in Multi-session Agents (ICLR 2026 · similarity 0.85)
- DialogXpert: Driving Intelligent and Emotion-Aware Conversations Through Online Value-Based Reinforcement Learning with LLM Priors (AAAI 2026 · similarity 0.84)
- DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search (ICLR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
