Research map / Architectures and efficiency / Attention and transformers
Coherent Dialogue with Attention-Based Language Models
Hongyuan Mei, Mohit Bansal, Matthew Walter
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use dynamic attention over the growing dialogue history | RNN language model · MovieTriples test | PPL: 27.09 -> 25.52 | Improved |
| Use dynamic attention over the growing dialogue history | RNN language model · Ubuntu Troubleshoot | recall@1: 11% -> 17% | Improved |
| Model the full dialogue as a language-model sequence rather than a source-target pair | Dialogue modeling · MovieTriples | PPL: 35.10 -> 27.81 | Improved |
| Model the full dialogue as a language-model sequence rather than a source-target pair | Dialogue modeling · Ubuntu Troubleshoot | PPL: 104.61 -> 56.16 | Improved |
| Add LDA topic features without reranking (ablation) | A-RNN · Ubuntu Troubleshoot test | BLEU: 0.3713 -> 0.3128 | Hurt |
| Rerank continuations using LDA topic matching | A-RNN · Ubuntu Troubleshoot test | BLEU: 0.3713 -> 0.4279 | Improved |
| Add LDA topic features and rerank using LDA topic matching | A-RNN · Ubuntu Troubleshoot test | BLEU: 0.3128 -> 0.3971 | Improved |
| Add LDA topic features without reranking (ablation) | A-RNN · Ubuntu Troubleshoot dev | BLEU: 0.2702 -> 0.2908 | Improved |
| Use dynamic attention instead of a plain RNN | RNN language model · Ubuntu Troubleshoot | Distinct-1: 0.0007 -> 0.0104 | Improved |
| Use dynamic attention instead of a plain RNN | RNN language model · MovieTriples | Distinct-1: 0.0004 -> 0.0028 | Improved |
Closest papers by meaning
- RAT: Bridging RNN Efficiency and Attention Accuracy via Chunk-based Sequence Modeling (NeurIPS 2025 · similarity 0.87)
- StreamingDialogue: Prolonged Dialogue Learning via Long Context Compression with Minimal Losses (NeurIPS 2024 · similarity 0.87)
- Aligning Spoken Dialogue Models from User Interactions (ICML 2025 · similarity 0.86)
- From Isolated Conversations to Hierarchical Schemas: Dynamic Tree Memory Representation for LLMs (ICLR 2025 · similarity 0.86)
- Simple linear attention language models balance the recall-throughput tradeoff (ICML 2024 · similarity 0.86)
- Structure-Aware Multimodal Sequential Learning for Visual Dialog (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
