Research map / Architectures and efficiency / Attention and transformers
Generalizable Multi-linear Attention Network
Tao Jin, Zhou Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove combinational addition operation (CAO) (ablation) | MAN (Non-Transformer) · CMU-MOSI | BA: 79.4 -> 78.4 | Improved |
| remove combinational addition operation (CAO) (ablation) | MAN (Transformer) · CMU-MOSI | BA: 82.7 -> 81.5 | Improved |
| remove local sequential constraints (LSC) (ablation) | MAN (Transformer) · CMU-MOSI | BA: 82.7 -> 81.4 | Improved |
| remove hierarchical approximation decomposition (HAD) (ablation) | MAN (Non-Transformer) · CMU-MOSI | BA: 79.4 -> 79.4 | No clear effect |
| remove hierarchical approximation decomposition (HAD) (ablation) | MAN (Transformer) · CMU-MOSI | BA: 82.7 -> 82.6 | No clear effect |
| remove HAD (high-order attention map) (ablation) | MAN · CMU-MOSI | FLOPs: 3.87 × 105 -> 10.23 × 106 | Hurt |
| remove CAO (ablation) | MAN · CMU-MOSI | FLOPs: 3.87 × 105 -> 8.31 × 106 | Hurt |
| remove LSC (ablation) | MAN · CMU-MOSI | FLOPs: 3.87 × 105 -> 3.07 × 105 | Improved |
| use Transformer backbone instead of non-Transformer | MAN · CMU-MOSI | BA: 79.4 -> 82.7 | Improved |
| add MAN module to LMF backbone, compare to LMF | MAN (Non-Transformer) · CMU-MOSI | MAE: 0.912 -> 0.894 | Improved |
| add MAN module to MulT backbone, compare to MulT | MAN (Transformer) · CMU-MOSI | BA / F1: 82.8 -> 83.0 | Mixed |
| add MAN module to MMT backbone, compare to MMT | MAN (Transformer) · MSR-VTT text-to-video | R@1: 24.6 -> 24.1 | No clear effect |
Closest papers by meaning
- MultiMax: Sparse and Multi-Modal Attention Learning (ICML 2024 · similarity 0.85)
- MetaLA: Unified Optimal Linear Approximation to Softmax Attention Map (NeurIPS 2024 · similarity 0.84)
- Degrees of Freedom for Linear Attention: Distilling Softmax Attention with Optimal Feature Efficiency (NeurIPS 2025 · similarity 0.84)
- Hierarchical Self-Attention: Generalizing Neural Attention Mechanics to Multi-Scale Problems (NeurIPS 2025 · similarity 0.84)
- MODA: MOdular Duplex Attention for Multimodal Perception, Cognition, and Emotion Understanding (ICML 2025 · similarity 0.84)
- A Multi-Focus-Driven Multi-Branch Network for Robust Multimodal Sentiment Analysis (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
