Research map / Label-efficient and robust learning / Recognition and long-tail
Identifying and Controlling Important Neurons in Neural Machine Translation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Rank neurons by cross-model correlation, then erase the top-ranked 10% | English-Spanish NMT, MaxCorr · English-Spanish test set | BLEU: 15-20 points | Hurt |
| Erase the bottom-ranked 10% of neurons instead of the top-ranked 10% | English-Spanish NMT, MaxCorr · English-Spanish test set | BLEU: 2-3 points | Hurt |
| Rank neurons by cross-model correlation, then erase the top-ranked 10% | English-Spanish NMT, MinCorr · English-Spanish test set | BLEU: 15-20 points | Hurt |
| Rank neurons by regression, then erase the top-ranked 10% | English-Spanish NMT, LinReg · English-Spanish test set | BLEU: 15-20 points | Hurt |
| Rank directions with SVCCA, then erase the top-ranked 1% | English-Spanish NMT, SVCCA · English-Spanish test set | BLEU: 15 BLEU point drop | Hurt |
| Use MaxCorr-ranked neurons to modify the most predictive tense neuron's activation | English-Spanish NMT · Tense control | Success rate: 67% | Helped |
| Modify the most predictive tense neuron's activation for present-to-past control | English-Spanish NMT · Tense control | Success rate: 49% | Helped |
| Modify the 5 top number neurons | English-Spanish NMT · Number control | Success rate: 37% | Helped |
| Modify the 5 top gender neurons | English-Spanish NMT · Gender control | Success rate: 21% | Helped |
| Modify more than the 5 top gender neurons | English-Spanish NMT · Gender control | Success rate | No clear effect |
| Use MaxCorr to rank neurons | NMT models · Noun phrase segmentation | Accuracy: 60-80% | Helped |
Closest papers by meaning
- What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Models (AAAI 2019 · similarity 0.89)
- Neuron Interaction Based Representation Composition for Neural Machine Translation (AAAI 2020 · similarity 0.89)
- From Fully Trained to Fully Random Embeddings: Improving Neural Machine Translation with Compact Word Embedding Tables (AAAI 2022 · similarity 0.88)
- Neural Machine Translation Advised by Statistical Machine Translation (AAAI 2017 · similarity 0.88)
- Unsupervised Neural Machine Translation (ICLR 2018 · similarity 0.87)
- Neurons to Words: A Novel Method for Automated Neural Network Interpretability and Alignment (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.