Research map / Architectures and efficiency / Attention and transformers
Multi-Mention Learning for Reading Comprehension with Neural Cascades
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use multi-loss training across the cascade rather than only the level 3 loss (ablation) | 3-Level Cascade · TriviaQA Wikipedia Dev Full | EM: 43.48 -> 52.18 | Helped |
| Combine the two level 1 submodels into one feed-forward network (ablation) | 3-Level Cascade · TriviaQA Wikipedia Dev Full | EM: 52.18 -> 52.25 | No clear effect |
| Add level 3 aggregation of repeated answer mentions (ablation) | 3-Level Cascade · TriviaQA Wikipedia Dev Full | EM: 46.52 -> 52.18 | Helped |
| Add level 2 question-to-sentence attention (ablation) | 3-Level Cascade · TriviaQA Wikipedia Dev Full | EM: 19.75 -> 46.52 | Helped |
| Use span-and-context features rather than question-and-span features alone (ablation) | Level 1 model · TriviaQA Wikipedia Dev Full | EM: 15.75 -> 19.75 | Helped |
| Use the full cascade instead of only the question-and-span submodel (ablation) | 3-Level Cascade · TriviaQA Wikipedia Dev Full | EM: 15.75 -> 52.18 | Helped |
| Use the full cascade instead of only the span-and-context submodel (ablation) | 3-Level Cascade · TriviaQA Wikipedia Dev Full | EM: 19.75 -> 52.18 | Helped |
| Process longer evidence with the cascaded feed-forward model rather than a recurrent baseline | Neural Cascades · TriviaQA document length 10,000 tokens | speedup: 45$\times$ | Helped |
| Use the cascaded model rather than the strongest listed leaderboard baseline | Neural Cascades · TriviaQA Wikipedia Full test | EM: 48.64 -> 51.59 | Helped |
| Use the cascaded model rather than the strongest listed leaderboard baseline | Neural Cascades · TriviaQA Web Verified test | F1: 67.97 -> 66.88 | Hurt |
Closest papers by meaning
- A Deep Cascade Model for Multi-Document Reading Comprehension (AAAI 2019 · similarity 0.89)
- Hierarchical Attention Flow for Multiple-Choice Reading Comprehension (AAAI 2018 · similarity 0.88)
- Co-Attention Hierarchical Network: Generating Coherent Long Distractors for Reading Comprehension (AAAI 2020 · similarity 0.88)
- Generating Distractors for Reading Comprehension Questions from Real Examinations (AAAI 2019 · similarity 0.87)
- Densely Connected Attention Propagation for Reading Comprehension (NeurIPS 2018 · similarity 0.86)
- Making Retrieval-Augmented Language Models Robust to Irrelevant Context (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.