Research map / Large language models / Instruction and fine-tuning
SAE as a Crystal Ball: Interpretable Features Predict Cross-domain Transferability of LLMs without Training
Qi Zhang, Yifei Wang, Xiaohan Wang, Jiajun Chai, Guojun Yin, Wei Lin, Yisen Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use STS_ICL instead of STS_act (activation-sum) to measure domain correlation with shifted dimensions (ablation) | STS · MMLU-Pro (SFT on LIMO) | Pearson correlation: Gemma2-9B 0.60 -> 0.77; Llama3-8B 0.71 -> 0.81; Qwen2.5-7B 0.90 -> 0.78 | Mixed |
| Use SAE features with higher sparsity (lower L0) (ablation) | STS on Gemma2-9B-Instruct · MMLU-Pro (SFT on LIMO) | Pearson correlation: L0 109: 0.65 -> L0 13: 0.78 | Improved |
| Use SAE with larger hidden dimension (131k vs 16k, stronger monosemanticity) (ablation) | STS · MMLU-Pro (SFT on LIMO) | Pearson correlation | Improved |
| Select dimensions with the largest shifts rather than smaller-shifted dimensions (ablation) | STS · MMLU-Pro (SFT on LIMO) | Pearson correlation | Improved |
| Apply SAE layer choice of 15, 20 or 25 instead of fixed layer (ablation) | STS on Qwen2.5-7B-Instruct · MMLU-Pro (SFT on LIMO) | Pearson correlation | No clear effect |
| Use SAE features instead of raw model features to find ICL-SFT shifted dimensions (ablation) | shifted dimension identification · LIMO SFT of Qwen2.5-7B-Instruct | overlap of top-100 shifted dims: 57% of the top 100 most shifted SAE dimensions coincide | Improved |
| Use STS instead of raw feature activations as predictor | transferability prediction · MMLU-Pro (Qwen SFT on LIMO) | correlation coefficient: 0.03 -> 0.79 | Improved |
| Use STS instead of SAE feature activation probe | transferability prediction · MMLU-Pro (Qwen SFT on LIMO) | correlation coefficient: 0.08 -> 0.79 | Improved |
| Use STS (no SFT model needed) instead of representation similarity before/after SFT | transferability prediction · MMLU-Pro (Qwen SFT on LIMO) | correlation coefficient: 0.61 -> 0.79 | Improved |
| Allocate STS-proportional extra data to engineering and law domains | SFT of Qwen2.5-7B-Instruct on LIMO · MMLU-Pro engineering and law | downstream accuracy | Improved |
| Use STS-guided data mixture instead of LIMO only | SFT of Qwen2.5-7B-Instruct · Math-LightEval | accuracy (%): 81.3 -> 80.1 | No clear effect |
| Fine-tune estimated shifted dimensions instead of random dimensions (3000 dims) (ablation) | selective fine-tuning · Math-LightEval | accuracy (%): 74.0 -> 76.9 | Improved |
Closest papers by meaning
- Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (ICML 2025 · similarity 0.89)
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability (ICML 2025 · similarity 0.88)
- Unveiling the Secret Recipe: A Guide For Supervised Fine-Tuning Small LLMs (ICLR 2025 · similarity 0.88)
- Transferring Linear Features Across Language Models With Model Stitching (NeurIPS 2025 · similarity 0.87)
- Measuring Progress in Dictionary Learning for Language Model Interpretability with Board Game Models (NeurIPS 2024 · similarity 0.87)
- Sparse Autoencoders Reveal Temporal Difference Learning in Large Language Models (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
