Research map / Large language models / Retrieval and knowledge
KPT: Keyword-Guided Pre-training for Grounded Dialog Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply KPT rather than response-generation pre-training alone | T5-base · MultiWOZ | Knowledge Utility: 64.2 -> 84.0 | Helped |
| Apply KPT rather than response-generation pre-training alone | T5-base · OpenDialKG | Knowledge Utility: 52.6 -> 63.6 | Helped |
| Apply KPT rather than response-generation pre-training alone | T5-base · PersonaChat | Knowledge Utility: 14.2 -> 20.1 | Helped |
| Apply KPT rather than response-generation pre-training alone | T5-base · WoW | Knowledge Utility: 5.1 -> 14.0 | Helped |
| Add response-generation pre-training before KPT | T5-KPT · Grounded dialog tasks | Few-shot performance | No clear effect |
| Use only golden keywords rather than both KPT objectives (ablation) | T5-RG-KPT · MultiWOZ | Unigram F1: 46.8/50.6/53.0 -> 48.0/50.7/52.5 | Helped |
| Add the noisy-keyword objective (ablation) | T5-RG-K_g · PersonaChat | Unigram F1: 7.7/7.4/7.5 -> 7.9/7.9/8.2 | Mixed |
| Use either single keyword objective rather than both (ablation) | T5-RG-KPT · OpenDialKG | Unigram F1: 32.7/35.2/34.6; K_g 32.7/33.9/35.0; K_n 33.8/33.7/33.8 | Mixed |
| Increase keyword ratio from 0.3 toward 1.0 (ablation) | T5-RG-KPT · MultiWOZ | Performance across tasks | Helped |
| Increase keyword ratio above 0.3 (ablation) | T5-RG-KPT · OpenDialKG, PersonaChat, and WoW | Performance across tasks | Hurt |
| Replace GPT-2 keyword extraction with random extraction (ablation) | T5-RG-KPT · PersonaChat and WoW | Knowledge Utility | Hurt |
| Replace GPT-2 keyword extraction with DialoGPT (ablation) | T5-RG-KPT · OpenDialKG and PersonaChat | Knowledge Utility | Hurt |
Closest papers by meaning
- Zero-Resource Knowledge-Grounded Dialogue Generation (NeurIPS 2020 · similarity 0.92)
- Low-Resource Knowledge-Grounded Dialogue Generation (ICLR 2020 · similarity 0.91)
- Improving Open-Domain Dialogue Response Generation with Multi-Source Multilingual Commonsense Knowledge (AAAI 2024 · similarity 0.90)
- Sequential Latent Knowledge Selection for Knowledge-Grounded Dialogue (ICLR 2020 · similarity 0.89)
- Improving the Robustness of Knowledge-Grounded Dialogue via Contrastive Learning (AAAI 2024 · similarity 0.88)
- From Retrieval to Generation: A Simple and Unified Generative Model for End-to-End Task-Oriented Dialogue (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.