Research map / Architectures and efficiency / Attention and transformers
Neuron Interaction Based Representation Composition for Neural Machine Translation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add NI-based composition to multi-layer and multi-head representations | Transformer-Base · WMT14 En→De | BLEU: 27.31 -> 28.54 | Helped |
| Add NI-based composition to multi-layer and multi-head representations | Transformer-Base · WMT14 En→Fr | BLEU: 39.28 -> 40.15 | Helped |
| Add NI-based composition to multi-layer and multi-head representations | Transformer-Big · WMT14 En→De | BLEU: 28.58 -> 29.17 | Helped |
| Add NI-based composition to multi-layer and multi-head representations | Transformer-Big · WMT14 En→Fr | BLEU: 41.41 -> 42.10 | Helped |
| Apply NI-based composition to both layers and heads rather than either alone | Transformer-Base · WMT14 En→De | BLEU: 28.31, 28.29 -> 28.54 | Helped |
| Use bilinear pooling without first-order features instead of linear combination (ablation) | NI-based composition · WMT14 En→De | BLEU | Helped |
| Add first-order features to bilinear pooling (ablation) | NI-based composition · WMT14 En→De | BLEU | Helped |
| Increase the bilinear pooling rank (ablation) | Multi-head NI composition · WMT14 En→De | BLEU | Helped |
| Reduce bilinear pooling rank to 32 (ablation) | Multi-head NI composition · WMT14 En→De | BLEU | Helped |
| Use NI composition for multi-layer or multi-head representations | Transformer-Base · WMT14 En→De sentence-length segments | BLEU | Helped |
Closest papers by meaning
- Identifying and Controlling Important Neurons in Neural Machine Translation (ICLR 2019 · similarity 0.89)
- Neural Machine Translation with Joint Representation (AAAI 2020 · similarity 0.87)
- Layer-Wise Representation Fusion for Compositional Generalization (AAAI 2024 · similarity 0.85)
- Towards Neuron Attributions in Multi-Modal Large Language Models (NeurIPS 2024 · similarity 0.85)
- Talking Heads: Understanding Inter-Layer Communication in Transformer Language Models (NeurIPS 2024 · similarity 0.85)
- Neurons to Words: A Novel Method for Automated Neural Network Interpretability and Alignment (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.