Research map / Large language models / Language, safety and interpretability
How Does Alignment Enhance LLMs’ Multilingual Capabilities? A Language Neurons Perspective
Shimao Zhang, Zhejian Lai, Xiang Liu, Shuaijie She, Xiao Liu, Yeyun Gong, Shujian Huang, Jiajun Chen
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| align with zh/de => en (MAPO DPO) | MistralMathOctopus-7B · MGSM (average) | accuracy: 57.8 -> 63.6 | Improved |
| align with sw/th => en (MAPO DPO) | MistralMathOctopus-7B · MGSM (average) | accuracy: 57.8 -> 64.7 | Improved |
| align with all 9 languages => en | MistralMathOctopus-7B · MGSM (average) | accuracy: 57.8 -> 62.0 | Improved |
| align with all 9 languages instead of sw/th | MistralMathOctopus-7B · MGSM (average) | accuracy: 64.7 -> 62.0 | Hurt |
| align with sw/th => en (MAPO DPO) | MistralMathOctopus-7B · MSVAMP (average) | accuracy: 64.4 -> 69.3 | Improved |
| align with zh/de => en (unseen languages) | MistralMathOctopus-7B · MGSM (sw) | accuracy: 50.4 -> 59.2 | Improved |
| align with zh => en | MistralMathOctopus-7B · MGSM (en) | accuracy: 75.6 -> 75.2 | No clear effect |
| align with zh/de => en: language-specific neurons change (trained langs) | MistralMathOctopus-7B · MGSM | language-specific neuron count: -37 | Hurt |
| align with zh/de => en: language-related neurons change (trained langs) | MistralMathOctopus-7B · MGSM | language-related neuron count: +232 | Improved |
| align with zh/de => en: language-related neurons (unseen langs) | MistralMathOctopus-7B · MGSM | language-related neuron count: +205 | Improved |
| deactivate language-specific plus language-related neurons instead of only specific (ablation) | MistralMathOctopus-7B · MGSM | accuracy/PPL change: more pronounced effect | Improved |
| deactivate English language neurons (ablation) | MistralMathOctopus-7B · MGSM (en) | accuracy | No clear effect |
Closest papers by meaning
- How do Large Language Models Handle Multilingualism? (NeurIPS 2024 · similarity 0.91)
- The Same but Different: Structural Similarities and Differences in Multilingual Language Modeling (ICLR 2025 · similarity 0.89)
- The Rise and Down of Babel Tower: Investigating the Evolution Process of Multilingual Code Large Language Model (ICLR 2025 · similarity 0.88)
- Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons (AAAI 2024 · similarity 0.87)
- LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformations (NeurIPS 2026 · similarity 0.87)
- The Emergence of Abstract Thought in Large Language Models Beyond Any Language (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
