Research map / Architectures and efficiency / Attention and transformers
Who Did They Respond to? Conversation Structure Modeling Using Masked Hierarchical Transformer
Henghui Zhu, Feng Nan, Zhiguo Wang, Ramesh Nallapati, Bing Xiang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ancestral conversation context with ancestor masking instead of pairwise utterance classification | Reddit-small parent prediction · Reddit-small | Graph accuracy: 55.86% -> 60.53% | Improved |
| Use ancestral conversation context with ancestor masking instead of pairwise utterance classification | Reddit-small conversation reconstruction · Reddit-small | Conversation accuracy: 17.78% -> 20.61% | Improved |
| Use ancestral conversation context with ancestor masking instead of pairwise utterance classification | Reddit-large parent prediction · Reddit-large | Graph accuracy: 39.48% -> 42.87% | Improved |
| Use ancestral conversation context with ancestor masking instead of selecting the title comment | Reddit-large conversation reconstruction · Reddit-large | Conversation accuracy: 0.06% -> 0.13% | Improved |
| Add hand-crafted features to the masked hierarchical transformer | Ubuntu IRC parent prediction · Ubuntu IRC test set | Graph F1: 52.8 -> 70.6 | Improved |
| Add hand-crafted features to the masked hierarchical transformer | Ubuntu IRC conversation reconstruction · Ubuntu IRC test set | Conversation F1: 10.3 -> 34.2 | Improved |
| Add ancestor masking so utterances attend to ancestors rather than all utterances (ablation) | Masked Hierarchical Transformer · Reddit-small | Graph accuracy: 46.43% -> 60.53% | Improved |
| Add ancestor masking so utterances attend to ancestors rather than all utterances (ablation) | Masked Hierarchical Transformer · Reddit-small | Conversation accuracy: 15.14% -> 20.61% | Improved |
| Increase the allowed ancestor depth (ablation) | Masked Hierarchical Transformer · Reddit-small | Graph accuracy | Improved |
| Use temporal-only masking rather than ancestor-based masking (ablation) | Masked Hierarchical Transformer · Reddit-small | Graph accuracy | Hurt |
| Increase temporal depth in temporal-only masking (ablation) | Masked Hierarchical Transformer · Reddit-small | Graph accuracy | Improved |
Closest papers by meaning
- SD-Eval: A Benchmark Dataset for Spoken Dialogue Understanding Beyond Words (NeurIPS 2024 · similarity 0.83)
- DialectLLM: A Dialect-Aware Dialog[ue] Generation Framework Beyond Standard American English (NeurIPS 2026 · similarity 0.83)
- Aligning Spoken Dialogue Models from User Interactions (ICML 2025 · similarity 0.82)
- Mutable Transcripts: Mitigating Context Pollution through Editable Conversation State (NeurIPS 2026 · similarity 0.82)
- SeCom: On Memory Construction and Retrieval for Personalized Conversational Agents (ICLR 2025 · similarity 0.82)
- Linear Decoding of Morphology Relations in Language Models (Student Abstract) (AAAI 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
