Research map / Architectures and efficiency / Attention and transformers
Multi-Level Head-Wise Match and Aggregation in Transformer for Textual Sequence Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace classic sequence-vector matching with single-level head-wise matching | BERT-large sequence matching · SNLI | accuracy: 83.2 -> 87.0 | Helped |
| Replace classic sequence-vector matching with single-level head-wise matching | BERT-large sequence matching · MNLI-m | accuracy: 73.8 -> 77.7 | Helped |
| Replace classic sequence-vector matching with single-level head-wise matching | BERT-large sequence matching · MNLI-mm | accuracy: 74.2 -> 77.4 | Helped |
| Replace classic sequence-vector matching with single-level head-wise matching | BERT-large sequence matching · QQP | F1: 63.3 -> 67.8 | Helped |
| Add multi-level head-wise representations to single-level head-wise matching | BERT-large sequence matching · SNLI | accuracy: 87.0 -> 88.1 | Helped |
| Add multi-level head-wise representations to single-level head-wise matching | BERT-large sequence matching · MNLI-m | accuracy: 77.7 -> 79.2 | Helped |
| Add multi-level head-wise representations to single-level head-wise matching | BERT-large sequence matching · MNLI-mm | accuracy: 77.4 -> 79.3 | Helped |
| Add multi-level head-wise representations to single-level head-wise matching | BERT-large sequence matching · SQuAD-binary | accuracy: 62.1 -> 62.9 | Helped |
| Use element-wise matching rather than cosine matching (ablation) | Multi-level head-wise matching · SNLI | accuracy: 86.8 -> 88.1 | Helped |
| Use element-wise matching rather than bilinear matching (ablation) | Multi-level head-wise matching · SNLI | accuracy: 86.7 -> 88.1 | Helped |
| Use max-pooling rather than concatenation aggregation (ablation) | Element-wise multi-level head-wise matching · SNLI | accuracy: 88.0 -> 88.1 | No clear effect |
| Remove hierarchical aggregation across layers (ablation) | Element-wise multi-level head-wise matching · SNLI | accuracy: 88.0 -> 87.6 | Hurt |
Closest papers by meaning
- A Compare-Aggregate Model for Matching Text Sequences (ICLR 2017 · similarity 0.86)
- How Transformers Learn Regular Language Recognition: A Theoretical Study on Training Dynamics and Implicit Bias (ICML 2025 · similarity 0.84)
- LET: Linguistic Knowledge Enhanced Graph Transformer for Chinese Short Text Matching (AAAI 2021 · similarity 0.84)
- Deconvolutional Latent-Variable Model for Text Sequence Matching (AAAI 2018 · similarity 0.84)
- RMIB: Representation Matching Information Bottleneck for Matching Text Representations (ICML 2024 · similarity 0.82)
- Fundamental Limitations on Subquadratic Alternatives to Transformers (ICLR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.