Research map / Large language models / Retrieval and knowledge
Bridging the Language Gap: Uncovering and Aligning Shared Circuits for Multi-Hop Reasoning in Multilingual LLMs
Chenghao Sun, Zhen Huang, Yonggang Zhang, Xinmei Tian, Xu Shen, Jieping Ye
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune only the top-40 critical attention heads | Qwen2.5-7B-Instruct · XHopReason (8-language average, trained on Japanese) | accuracy (%): 15.34 -> 18.64 | Improved |
| fine-tune all parameters instead of critical heads (ablation) | Qwen2.5-7B-Instruct · XHopReason (8-language average) | accuracy (%): 15.34 -> 15.17 | No clear effect |
| apply DPO instead of critical-head tuning (ablation) | Qwen2.5-7B-Instruct · XHopReason (8-language average) | accuracy (%): 15.34 -> 15.98 | No clear effect |
| tune 100 least critical heads rather than top critical heads (ablation) | head tuning · XHopReason (8-language average) | accuracy (%): 18.64 -> 17.58 | Hurt |
| tune top-20 heads instead of top-40 (ablation) | head tuning · XHopReason (8-language average) | accuracy (%): 18.64 -> 18.16 | Hurt |
| tune top-80 or top-160 heads instead of top-40 (ablation) | head tuning · XHopReason (8-language average) | accuracy (%): 18.64 -> 18.39 (top-80), 18.01 (top-160) | Hurt |
| train head tuning on a single source language, test on other languages | Top-40 heads MFT · XHopReason cross-lingual transfer | accuracy (%): Italian 19.96 -> 22.05 when trained on Japanese | Improved |
| fine-tune all parameters instead of critical heads (ablation) | Qwen2.5-7B-Instruct · General benchmarks (MMLU, StrategyQA, INCLUDE, Belebele) | average change (points): -24.58 average change | Hurt |
| apply DPO instead of critical-head tuning (ablation) | Qwen2.5-7B-Instruct · General benchmarks | average change (points): -22.72 average change | Hurt |
| fine-tune only top-40 heads | Qwen2.5-7B-Instruct · General benchmarks | average change (points): -0.85 average change | No clear effect |
| fine-tune only top-40 heads | Qwen2.5-7B-Instruct · StrategyQA (en) | accuracy (%): 56.83 -> 61.13 | Improved |
| fine-tune only top-40 heads | Qwen2.5-7B-Instruct · Japanese test accuracy | accuracy (%): Top-40 20.28 vs Full Fine-tuning 22.35 and DPO 24.10 | Mixed |
Closest papers by meaning
- AdaMCoT: Rethinking Cross-Lingual Factual Reasoning Through Adaptive Multilingual Chain-of-Thought (AAAI 2026 · similarity 0.91)
- Journey to the Center of the Knowledge Neurons: Discoveries of Language-Independent Knowledge Neurons and Degenerate Knowledge Neurons (AAAI 2024 · similarity 0.91)
- When Less Language is More: Language-Reasoning Disentanglement Makes LLMs Better Multilingual Reasoners (NeurIPS 2025 · similarity 0.90)
- Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought (ICLR 2026 · similarity 0.88)
- Long Chain-of-Thought Reasoning Across Languages (ICLR 2026 · similarity 0.88)
- How do Large Language Models Handle Multilingualism? (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
