Research map / Large language models / Retrieval and knowledge
From Retrieval to Generation: A Simple and Unified Generative Model for End-to-End Task-Oriented Dialogue
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Uni-ToD recasts end-to-end task-oriented dialogue as a single GPT-2 causal language modeling task that first generates knowledge base records under a prefix-trie constraint and then generates the response.
Problem
End-to-end task-oriented dialogue systems retrieve knowledge base records with separate retrievers or memory networks, which decouples retrieval from generation and makes joint optimization difficult.
Approach
Treat knowledge retrieval as constrained generation: a prefix trie built from the KB restricts decoding so the language model generates only existing records, then generates the response, all trained with a single maximum-likelihood loss.
Key results
- On CamRest, Uni-ToD reaches 24.7 BLEU and 77.8 Entity F1, versus 23.4 and 75.6 for the previous best DialoKG.
- On In-Car Assistant, Uni-ToD reaches 22.0 BLEU and 66.6 Entity F1, versus 20.0 and 65.9 for DialoKG.
- On the large-scale CamRest KB, Uni-ToD gets 21.2 BLEU and 72.3 Entity F1, the best Entity F1 among the compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Uni-ToD instead of DialoKG | EToD systems · CamRest | BLEU: 23.4 -> 24.7* | Helped |
| use Uni-ToD instead of DialoKG | EToD systems · CamRest | Entity F1: 75.6 -> 77.8* | Helped |
| use Uni-ToD instead of DialoKG | EToD systems · In-Car Assistant | BLEU: 20.0 -> 22.0* | Helped |
| use Uni-ToD instead of DialoKG | EToD systems · In-Car Assistant | Entity F1: 65.9 -> 66.6* | Helped |
| use Uni-ToD instead of GPT2+KE | EToD systems · Multi-WOZ 2.1 | BLEU: 15.0 -> 12.3* | Hurt |
| use Uni-ToD instead of DialoKG | EToD systems · Multi-WOZ 2.1 | Entity F1: 43.5 -> 44.3* | Helped |
| add prefix trie constraint to record generation (ablation) | Uni-ToD · CamRest | BLEU: 17.2 -> 24.7 | Helped |
| add prefix trie constraint to record generation (ablation) | Uni-ToD · In-Car Assistant | Entity F1: 49.0 -> 66.6 | Helped |
| replace trie-constrained record generation with concatenated KB input (ablation) | Uni-ToD · CamRest | BLEU: 24.7 -> 12.7 | Hurt |
| replace trie-constrained record generation with concatenated KB input (ablation) | Uni-ToD · In-Car Assistant | Entity F1: 66.6 -> 59.2 | Hurt |
| vary prefix trie ordering (ablation) | Uni-ToD · CamRest | BLEU: 24.7; 23.5; 24.4 | No clear effect |
| vary prefix trie ordering (ablation) | Uni-ToD · CamRest | Entity F1: 77.8; 78.1; 77.6 | No clear effect |
Surprising findings
- On In-Car, generated records are only 77.8% accurate, and oracle records give a notable gain (Entity F1 84.3 vs 66.6), so record generation is the main bottleneck.
- Removing the prefix trie and generating records freely drops CamRest Entity F1 from 77.8 to 65.5.
Limitations
- Generated KB records are not fully accurate (77.8% on In-Car), limiting response quality.
- Evaluated only on three small benchmark datasets with GPT-2 as the backbone; the paper states it has not yet explored fusing KBs into larger language models.
Experimental setup
Datasets: CamRest In-Car Assistant Multi-WOZ 2.1
Benchmarks: CamRest In-Car Assistant Multi-WOZ 2.1
Metrics: BLEU Entity F1 human rating (correct, fluent, humanlike)
Baselines: Mem2Seq GLMP DDMN GPT2+KE TTOS DF-Net EER FG2Seq MCL CDNet ECO GraphMemDialog
Builds on: GPT-2 DialoGPT De Cao et al. 2021 (autoregressive entity retrieval)
Closest papers by meaning
- Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models (AAAI 2016 · similarity 0.89)
- Improving Knowledge-Aware Dialogue Generation via Knowledge Base Question Answering (AAAI 2020 · similarity 0.89)
- Local Explanation of Dialogue Response Generation (NeurIPS 2021 · similarity 0.89)
- TA&AT: Enhancing Task-Oriented Dialog with Turn-Level Auxiliary Tasks and Action-Tree Based Scheduled Sampling (AAAI 2024 · similarity 0.87)
- Improving Open-Domain Dialogue Response Generation with Multi-Source Multilingual Commonsense Knowledge (AAAI 2024 · similarity 0.87)
- Divide-and-Conquer: Tree-structured Strategy with Answer Distribution Estimator for Goal-Oriented Visual Dialogue (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.