Research map / Architectures and efficiency / Attention and transformers
FusionNet: Fusing via Fully-aware Attention with Application to Machine Comprehension
Hsin-Yuan Huang, Chenguang Zhu, Yelong Shen, Weizhu Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FusionNet instead of the strongest listed single-model baseline | SQuAD hidden test · SQuAD hidden test | F1: 83.5 -> 83.9 | Improved |
| Use FusionNet instead of the strongest listed ensemble baseline | SQuAD hidden test · SQuAD hidden test | F1: 85.6 -> 85.9 | Improved |
| Use FusionNet (ensemble) instead of the strongest reported baseline | AddSent · AddSent | F1: 46.6 -> 51.4 | Improved |
| Use FusionNet (ensemble) instead of the strongest reported baseline | AddOneSent · AddOneSent | F1: 56.0 -> 60.7 | Improved |
| Add ReLU to symmetric attention scoring (ablation) | Fully-aware attention · SQuAD dev | F1: 81.5 -> 83.6 | Improved |
| Replace standard high-level attention with fully-aware attention (ablation) | FusionNet attention · SQuAD dev | F1: 73.2 -> 81.4 | Improved |
| Fuse each history-of-word level with its own attention weights rather than one shared weight (ablation) | Fully-aware multi-level fusion · SQuAD dev | F1: 80.7 -> 82.7 | Improved |
| Add standard self-boosted fusion (ablation) | FA Multi-Level · SQuAD dev | F1: 82.7 -> 82.6 | No clear effect |
| Replace standard self-boosted attention with fully-aware self-attention (ablation) | FA Multi-Level · SQuAD dev | F1: 82.6 -> 83.6 | Improved |
| Remove CoVe embeddings (ablation) | FusionNet · SQuAD dev | F1: 83.6 -> 82.5 | Hurt |
| Fix GloVe embeddings rather than fine-tuning them (ablation) | FusionNet · SQuAD dev | F1: 83.6 -> 83.2 | Hurt |
| Remove CoVe embeddings (ablation) | FusionNet · AddSent | F1: 50.1 -> 52.4 | Improved |
Later papers that compared with this method
5 later accepted papers report FusionNet as a baseline in their results tables; 5 of them report a better number on most of the results they share with it.
- Regulating Rather than Constraining: Adaptive Guidance for Complex Spectral Reconstruction in Pansharpening (CVPR 2026 · better on 3 of 3 shared results)
- Content-Adaptive Non-Local Convolution for Remote Sensing Pansharpening (CVPR 2024 · better on 1 of 1 shared result)
- LAGConv: Local-Context Adaptive Convolution Kernels with Global Harmonic Bias for Pansharpening (AAAI 2022 · better on 14 of 14 shared results)
- Dynamic Cross Feature Fusion for Remote Sensing Pansharpening (ICCV 2021 · better on 12 of 13 shared results)
- CubeNet: Equivariance to 3D Rotation and Translation (ECCV 2018 · better on 2 of 2 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- A Multi-View Fusion Neural Network for Answer Selection (AAAI 2018 · similarity 0.87)
- A Question-Focused Multi-Factor Attention Network for Question Answering (AAAI 2018 · similarity 0.86)
- Improved Fusion of Visual and Language Representations by Dense Symmetric Co-Attention for Visual Question Answering (CVPR 2018 · similarity 0.86)
- Bidirectional Attention Flow for Machine Comprehension (ICLR 2017 · similarity 0.85)
- Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Features (ICML 2024 · similarity 0.85)
- SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
