Research map / Large language models / Instruction and fine-tuning
Finding the Translation Switch: Discovering and Exploiting the Task-Initiation Features in LLMs
Xinwei Wu, Heng Liu, Xiaohu Zhao, Yuqi Ren, Linlong Xu, Longyue Wang, Deyi Xiong, Weihua Luo, Kaifu Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| amplify translation-initiation feature l12-f2291 (x2) | Gemma-2-2B-IT · WMT24++ en-zh | Hallucination rate (%): 19.15% -> 10.42% | Improved |
| amplify translation-initiation feature l12-f2291 | Gemma-2-2B-IT · WMT24++ en-ar | COMET: 40.52 -> 42.02 | Improved |
| ablate high-consistency features (layer 25, coeff 0) (ablation) | Gemma-2-2B-IT · WMT24++ en-zh | Hallucination rate (%): +47.99% hallucination, -8.49% COMET | Hurt |
| ablate low-consistency features (ablation) | Gemma-2-2B-IT · WMT24++ en-zh | Hallucination rate (%) | No clear effect |
| select fine-tuning data by low initiation-feature activation (S3) vs random (ablation) | Gemma-2-2B-IT · WMT24 en-zh fine-tuning (20k) | Hallucination rate (%): 3.62% -> 0.90% | Improved |
| select data by low initiation-feature activation (S3) vs high-quality (S1) (ablation) | Gemma-2-2B-IT · WMT24 en-zh fine-tuning (20k) | COMET: 83.32 -> 83.37 | No clear effect |
| select data by S3 vs high-loss (S2) (ablation) | LLaMA-3.1-1B-IT · WMT24 en-zh fine-tuning (20k) | COMET: 73.41 -> 77.92 | Improved |
| transfer Gemma-2-2B features to select data (S3 vs random) (ablation) | Gemma-2-9B-IT · WMT24 en-zh fine-tuning (50k, features from Gemma-2-2B) | COMET: 85.36 -> 86.48 | Improved |
| transfer Gemma-2-2B features to select data (S3 vs S1) (ablation) | LLaMA-3.2-8B-IT · WMT24 en-zh fine-tuning (50k, features from Gemma-2-2B) | COMET: 86.69 -> 86.34 | No clear effect |
| S3 selection with 50% of data (ablation) | Gemma-2-2B-IT · WMT24 en-zh fine-tuning (50% data) | COMET: 83.58 -> 83.68 | Improved |
Closest papers by meaning
- Exploring the Translation Mechanism of Large Language Models (NeurIPS 2025 · similarity 0.91)
- Remarkable Robustness of LLMs: Stages of Inference? (NeurIPS 2025 · similarity 0.89)
- A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models (ICLR 2024 · similarity 0.89)
- Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention (AAAI 2024 · similarity 0.88)
- When Scaling Meets LLM Finetuning: The Effect of Data, Model and Finetuning Method (ICLR 2024 · similarity 0.88)
- LLM Augmented LLMs: Expanding Capabilities through Composition (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
