AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

FusionNet: Fusing via Fully-aware Attention with Application to Machine Comprehension

Hsin-Yuan Huang, Chenguang Zhu, Yelong Shen, Weizhu Chen

ICLR 2018 · Architectures and efficiency › Attention and transformers · approach: Sparse and structured attention · 130 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use FusionNet instead of the strongest listed single-model baselineSQuAD hidden test · SQuAD hidden testF1: 83.5 -> 83.9Improved
Use FusionNet instead of the strongest listed ensemble baselineSQuAD hidden test · SQuAD hidden testF1: 85.6 -> 85.9Improved
Use FusionNet (ensemble) instead of the strongest reported baselineAddSent · AddSentF1: 46.6 -> 51.4Improved
Use FusionNet (ensemble) instead of the strongest reported baselineAddOneSent · AddOneSentF1: 56.0 -> 60.7Improved
Add ReLU to symmetric attention scoring (ablation)Fully-aware attention · SQuAD devF1: 81.5 -> 83.6Improved
Replace standard high-level attention with fully-aware attention (ablation)FusionNet attention · SQuAD devF1: 73.2 -> 81.4Improved
Fuse each history-of-word level with its own attention weights rather than one shared weight (ablation)Fully-aware multi-level fusion · SQuAD devF1: 80.7 -> 82.7Improved
Add standard self-boosted fusion (ablation)FA Multi-Level · SQuAD devF1: 82.7 -> 82.6No clear effect
Replace standard self-boosted attention with fully-aware self-attention (ablation)FA Multi-Level · SQuAD devF1: 82.6 -> 83.6Improved
Remove CoVe embeddings (ablation)FusionNet · SQuAD devF1: 83.6 -> 82.5Hurt
Fix GloVe embeddings rather than fine-tuning them (ablation)FusionNet · SQuAD devF1: 83.6 -> 83.2Hurt
Remove CoVe embeddings (ablation)FusionNet · AddSentF1: 50.1 -> 52.4Improved

Later papers that compared with this method

5 later accepted papers report FusionNet as a baseline in their results tables; 5 of them report a better number on most of the results they share with it.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy