Research map / Large language models / Instruction and fine-tuning
Enhancing Non-English Capabilities of English-Centric Large Language Models Through Deep Supervision Fine-Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes deep supervision fine-tuning (DFT) that adds logits- or feature-based losses at bottom (language conversion) and middle (English thinking) layers of English-centric LLMs, with entropy-based critical layer selection, to improve non-English performance.
Problem
English-centric LLMs perform much worse in non-English languages due to imbalanced training data, and their internal English-pivot workflow lacks explicit cross-lingual supervision in intermediate layers.
Approach
Adds intermediate-layer supervision (logits-based or feature-based) to instruction tuning: at bottom layers to convert target-language input into English representations, and at middle layers to constrain English reasoning, with critical layers chosen via an entropy-drop heuristic.
Key results
- On LLaMA-2-7b, DFT-logits/feature improve over TFT on multilingual QA; e.g., XQUAD Vietnamese F1 29.65 (DFT-logits) vs 27.36 (TFT).
- On Gemma-2-2b, DFT-feature reaches 22.78 F1 on XQUAD Vietnamese vs 17.58 for TFT.
- Translation en-zh COMET improves from 68.37 (TFT) to 75.62 with feature-based language conversion supervision, while logits-based supervision drops it to 36.79.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply DFT-logits instead of TFT | LLaMA-2-7B · XQUAD zh | F1: 23.41 -> 24.77 | Helped |
| Apply DFT-logits instead of TFT | LLaMA-2-7B · MLQA zh | F1: 23.86 -> 25.68 | Helped |
| Apply DFT-logits instead of SFT | LLaMA-2-7B · TruthfulQA zh | MC1: 30.49 -> 28.93 | Hurt |
| Apply DFT-feature instead of TFT | LLaMA-2-7B · MKQA zh | F1: 40.70 -> 40.90 | Helped |
| Apply DFT-feature instead of SDRRL | LLaMA-2-7B · XCOPA zh | accuracy: 65.20 -> 68.60 | Helped |
| Apply DFT-logits instead of TFT | Gemma-2-2B · MLQA zh | F1: 18.26 -> 20.13 | Helped |
| Apply DFT-feature instead of SFT | Gemma-2-2B · XNLI zh | accuracy: 40.36 -> 39.74 | Hurt |
| Apply DFT-logits instead of SFT | Gemma-2-2B · XStoryCloze zh | accuracy: 68.23 -> 66.91 | Hurt |
| Add logits-based English Thinking supervision (ablation) | TFT on LLaMA-2-7B · XQUAD zh | F1: 23.41 -> 24.47 | Helped |
| Add logits-based English Thinking supervision (ablation) | TFT on LLaMA-2-7B · TruthfulQA zh | MC1: 26.40 -> 28.93 | Helped |
| Add logits-based language conversion supervision (ablation) | TFT on LLaMA-2-7B · XNLI zh | accuracy: 36.87 -> 36.27 | Hurt |
| Add feature-based language conversion supervision (ablation) | TFT on LLaMA-2-7B · XCOPA zh | accuracy: 65.00 -> 66.80 | Helped |
Surprising findings
- Logits-based language-conversion supervision causes a catastrophic drop in en-zh translation (68.37 to 36.79 COMET).
- Plain SFT on English-only data outperforms target-language fine-tuning on TruthfulQA MC1, suggesting generation ability in the target language matters less for that metric.
Limitations
- Entropy-based critical layer selection is a rough heuristic that ignores token-level variation.
- Logits-based supervision harms some tasks and cross-lingual generation (translation).
- Evaluated only on Chinese, Vietnamese, and Arabic with two base models of 2B and 7B size.
Experimental setup
Datasets: Stanford Alpaca translated Alpaca (zh, vi, ar) FLORES-200
Benchmarks: XQUAD MLQA MKQA TruthfulQA XNLI XCOPA XStoryCloze MMLU FLORES-200
Metrics: F1 MC1 accuracy accuracy COMET
Baselines: SFT TFT SDRRL
Builds on: deep supervision networks instruction tuning SDRRL
Closest papers by meaning
- How do Large Language Models Handle Multilingualism? (NeurIPS 2024 · similarity 0.90)
- When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method (ICLR 2024 · similarity 0.89)
- A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models (ICLR 2024 · similarity 0.89)
- Tuning LayerNorm in Attention: Towards Efficient Multi-Modal LLM Finetuning (ICLR 2024 · similarity 0.89)
- Supervised Knowledge Makes Large Language Models Better In-context Learners (ICLR 2024 · similarity 0.89)
- MindMerger: Efficiently Boosting LLM Reasoning in non-English Languages (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.