Research map / Architectures and efficiency / RNNs and normalization
Breaking the Block: Preserving Data Continuity to Train Superior SAEs for Instruct Models
Jiaming Li, Haoran Ye, Yukun Chen, Xinyue Li, Lei Zhang, Hamid Alinejad-Rokny, Jimmy Chih-Hsien Peng, Min Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train SAE sequentially on independent instances (FAST) instead of concatenated blocks | SAE training on instruct models · Qwen2.5-7B-Instruct L18 8X Standard | log2 MSEst: 5.1985 -> 0.6468 | Improved |
| use FAST instead of block training | JumpReLU SAE · special-token reconstruction | log2 MSEst: -4.0005 -> -9.7604 | Improved |
| use FAST instead of block training | SAE training · Delta Loss | Delta Loss: -0.51% to 0.37% vs deviations up to 51.43% | Improved |
| use FAST instead of block training | SAE training · Llama-3.1-8B early training | GSNR: 0.26 -> 0.50 | Improved |
| use FAST instead of block training | SAE feature quality · Llama3.2-3B-Instruct | high-quality features %: 7.0% / 10.2% -> 21.1% | Improved |
| limit FAST truncation to 2048 tokens to match block context (ablation) | FAST · Llama-3.1-8B-Instruct L20 8X JumpReLU | log2(MSE): -15.56 -> -15.42 | No clear effect |
| match context length at 2048 tokens, FAST vs BT(F) (ablation) | FAST · Llama-3.1-8B-Instruct L20 8X JumpReLU | KL divergence: 8.19e-05 -> 2.62e-05 | Improved |
| prepend a random 512-token unrelated prefix to each FAST instance (RandomConcat) (ablation) | FAST · early training Steps 0-100 | GSNR | Hurt |
| steer special-token features with moderate scaling (alpha 15-50) | FAST JumpReLU SAE 8X · 1,010 instruction instances | generation quality | Improved |
| steer special-token features with excessive scaling (alpha > 100) (ablation) | FAST JumpReLU SAE 8X · 1,010 instruction instances | generation quality | Hurt |
Closest papers by meaning
- Improving Sparse Decomposition of Language Model Activations with Gated Sparse Autoencoders (NeurIPS 2024 · similarity 0.89)
- Low-Rank Adapting Models for Sparse Autoencoders (ICML 2025 · similarity 0.89)
- Sanity Checks for Sparse Autoencoders: Do SAEs Beat Random Baselines? (NeurIPS 2026 · similarity 0.89)
- Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders (ICML 2025 · similarity 0.89)
- Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (ICML 2025 · similarity 0.89)
- SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
