Research map / Architectures and efficiency / Attention and transformers
Ask Me Anything: Dynamic Memory Networks for Natural Language Processing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multiple episodic memory passes (5 vs 1) over inputs (ablation) | DMN · bAbI task 3 (three supporting facts) | test accuracy (%): 0 to 95.2 | Helped |
| add multiple episodic memory passes (5 vs 1) over inputs (ablation) | DMN · bAbI task 7 (counting) | test accuracy (%): 48.8 to 96.9 | Helped |
| increase episodic memory passes from 1 to 2 (ablation) | DMN · SST fine-grained | test accuracy (%): 51.5 to 52.1 | Helped |
| increase episodic memory passes from 2 to 3 (ablation) | DMN · SST fine-grained | test accuracy (%): 52.1 to 50.1 | Hurt |
| add episodic memory passes (0 to 2) over a no-memory baseline (ablation) | DMN · bAbI task 8 (lists/sets) | test accuracy (%): 33.6 to 55.6 at 2 passes; 96.5 at 5 passes | Helped |
| use DMN with episodic memory instead of MemNN | bAbI tasks 7 and 8 · bAbI | test accuracy (%): +11.9 on task 7; +5.5 on task 8 | Helped |
| use DMN instead of MemNN on bAbI overall | DMN · bAbI (20 tasks) | mean test accuracy (%): 93.3 to 93.6 | Helped |
| use DMN instead of MemNN on bAbI tasks 2 and 3 (long inputs) | DMN · bAbI tasks 2-3 | test accuracy (%): -1.8 on task 2; -4.8 on task 3 | Hurt |
| use softmax-based episode computation over gates instead of GRU-based episode (sentence selection) | DMN · bAbI | qualitative | Helped |
| use DMN with episodic memory instead of Kim CNN-MC on SST binary | DMN · SST binary | test accuracy (%): +0.5 over CNN-MC (88.1) | Helped |
| use DMN on SST fine-grained vs CT-LSTM | DMN · SST fine-grained | test accuracy (%): +1.1 over CT-LSTM (51.0) | Helped |
| use DMN on WSJ-PTB POS tagging | DMN · WSJ-PTB POS tagging | test accuracy (%): +0.06 over SCNN (97.50) | Helped |
Closest papers by meaning
- Dynamic Memory Networks for Visual and Textual Question Answering (ICML 2016 · similarity 0.92)
- Dynamic Coattention Networks For Question Answering (ICLR 2017 · similarity 0.88)
- Focused Hierarchical RNNs for Conditional Sequence Processing (ICML 2018 · similarity 0.88)
- Cross Temporal Recurrent Networks for Ranking Question Answer Pairs (AAAI 2018 · similarity 0.86)
- Question-Answering with Grammatically-Interpretable Representations (AAAI 2018 · similarity 0.86)
- Switchable Decision: Dynamic Neural Generation Networks (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.