AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Less is Enough: Synthesizing Diverse Data in LLM Feature Space with Sparse Autoencoders

Zhongzhi Li, Xuansheng Wu, Yijiang Li, Lijie Hu, Ninghao Liu

ICML 2026 · Large language models › Language, safety and interpretability · approach: Pruning and circuits in LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use FAC Synthesis instead of the strongest synthesis baseline (SynAlign)Llama-3.1-8B-Instruct · Toxicity DetectionAUPRC (%): 58.83 -> 62.60Improved
use FAC Synthesis instead of the best baseline on instruction followingLlama-3.1-8B-Instruct · AlpacaEval 2LC win rate (%): 11.26 -> 20.27Improved
increase FAC (task-relevant feature coverage) of the synthetic datasynthesis methods · Toxicity DetectionAUPRC (%): Pearson r = 0.95, Spearman 0.90Improved
cover a larger share of the missing features (30% to 100%) (ablation)FAC Synthesis · Toxicity DetectionAUPRC (%)Improved
raise sample count to a fixed N=200 at the same feature coverage (ablation)FAC Synthesis · Toxicity DetectionAUPRC (%): +0.50%No clear effect
use two-step contrastive-pair synthesis instead of one-step direct prompting (ablation)FAC Synthesis · Toxicity DetectionFAC (%)Improved
share synthetic data built from Llama-3.1-8B-Instruct features to fine-tune other model familiesMistral-7B-Instruct · Toxicity DetectionAUPRC (%): 27.66 -> 47.23Improved
share synthetic data built from Llama-3.1-8B-Instruct features to fine-tune Qwen2-7B-InstructQwen2-7B-Instruct · Toxicity DetectionAUPRC (%): 51.44 -> 68.20Improved
take SAE features from Llama-3.1-8B-Instruct instead of Qwen2's own SAE features (ablation)Qwen2-7B-Instruct downstream · Toxicity DetectionAUPRC (%): +1.60% to +5.13%Improved
use Llama-3.1-8B-Instruct as generator instead of GPT-4o mini (ablation)FAC Synthesis · Toxicity DetectionAUPRC (%): 44.90 -> 49.12 at temperature 0.8Improved
move decoding temperature away from the intermediate value 0.8 (ablation)FAC Synthesis · Toxicity DetectionAUPRC (%): 49.12 (T=0.8) vs 46.71 (T=0.4), 46.40 (T=1.2)Hurt
raise SAE activation threshold from 1.0 to 2.0 (ablation)FAC Synthesis · Toxicity DetectionAUPRC (%): 44.20 -> 45.98Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy