AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

SASFT: Sparse Autoencoder-guided Supervised Finetuning to Mitigate Unexpected Code-Switching in LLMs

Boyi Deng, Yu Wan, Baosong Yang, Fei Huang, Wenjie Wang, Fuli Feng

ICLR 2026 · Large language models › Instruction and fine-tuning · approach: Efficient LLM fine-tuning systems · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add SAE-guided auxiliary loss on language featuresSFT · code-switching any->ko, Qwen3-1.7B-Base 210kcode-switching ratio (%): 0.36 -> 0.00Improved
add SAE-guided auxiliary loss on language featuresSFT · code-switching any->zh, Gemma-2-9B 210kcode-switching ratio (%): 0.78 -> 0.41Improved
add SAE-guided auxiliary loss on language featuresSFT · code-switching any->zh, Qwen3-8B-Base 210kcode-switching ratio (%): 0.96 -> 0.66Improved
use SASFT instead of GRPO language-consistency rewardSFT+GRPO · code-switching, five models and three languagescode-switching ratio (%): Llama-3.1-8B ko 110k: GRPO 0.95 (+157%) vs SASFT 0.20Improved
set alpha to pre-estimated average instead of zero (ablation)SASFT · code-switching any->ru, Gemma-2-2B 210kcode-switching ratio (%): 0.90 -> 0.22Improved
apply SASFT to multiple layers instead of one (ablation)SASFT · code-switching ratio across modelscode-switching ratio (%)Improved
apply SASFT to multiple features instead of one (ablation)SASFT · code-switching to Chinesecode-switching ratio (%)Improved
apply SASFT to earlier single layer (ablation)SASFT · code-switching ratio, single layercode-switching ratio (%)Hurt
add SASFTSFT · MMLU, Llama-3.1-8Baccuracy (%): 29.99 -> 33.12Improved
add SASFTSFT · HumanEval, Llama-3.1-8Baccuracy (%): 87.74 -> 91.88Improved
add SASFTSFT · MMLU, Qwen3-8B-Baseaccuracy (%): 52.15 -> 50.09Hurt
add SASFTSFT · LogiQA, Gemma-2-2Baccuracy (%): 28.00 -> 25.75Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy