Research map / Architectures and efficiency / Attention and transformers
QANet: Combining Local Convolution with Global Self-Attention for Reading Comprehension
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace recurrent encoders with convolution and self-attention encoders | QANet on SQuAD · SQuAD | training speed: 2.9x; 9.4x; 13.3x | Helped |
| Combine convolution with self-attention rather than using self-attention alone | QANet · SQuAD | F1: +2.7 F1 | Helped |
| Add stochastic depth | QANet · SQuAD | F1: 0.2 F1 | Helped |
| Add backtranslation data augmentation at ratio 1:1:0 | QANet · SQuAD dev | F1: 82.7 -> 83.2; +0.5 | Helped |
| Add more French-pivoted augmented data beyond ×2 | QANet · SQuAD dev | F1 | No clear effect |
| Add German-pivoted backtranslation examples | QANet · SQuAD dev | F1: +0.2 | Helped |
| Increase augmented-data sampling weight to ratio 1:2:1 (ablation) | QANet with ×3 augmentation · SQuAD dev | F1: 83.4 -> 83.1; -0.3 | Hurt |
| Increase original-data sampling weight from ratio 1:2:1 to 2:2:1 (ablation) | QANet with ×3 augmentation · SQuAD dev | F1: +0.5 | Helped |
| Remove encoder convolutions (ablation) | QANet · SQuAD dev | F1: 82.7 -> 80.0; -2.7 | Hurt |
| Remove encoder self-attention (ablation) | QANet · SQuAD dev | F1: 82.7 -> 81.4; -1.3 | Hurt |
| Replace separable convolutions with normal convolutions (ablation) | QANet · SQuAD dev | F1: 82.7 -> 82.0; -0.7 | Hurt |
| Use a 3:1:1 ratio of original, English-French-English, and English-German-English examples (ablation) | QANet with ×3 augmentation · SQuAD dev | F1: 82.7 -> 83.8; +1.1 | Helped |
Closest papers by meaning
- Translucent Answer Predictions in Multi-Hop Reading Comprehension (AAAI 2020 · similarity 0.85)
- From Good to Best: Two-Stage Training for Cross-Lingual Machine Reading Comprehension (AAAI 2022 · similarity 0.85)
- Beyond RNNs: Positional Self-Attention with Co-Attention for Video Question Answering (AAAI 2019 · similarity 0.85)
- TAIA: Large Language Models are Out-of-Distribution Data Learners (NeurIPS 2024 · similarity 0.84)
- Rephrase, Augment, Reason: Visual Grounding of Questions for Vision-Language Models (ICLR 2024 · similarity 0.84)
- RAT: Bridging RNN Efficiency and Attention Accuracy via Chunk-based Sequence Modeling (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.