Research map / Architectures and efficiency / Pruning and efficient tuning
CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Features
Seonglae Cho, Zekun Wu, Adriano Koshiyama
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| select SAE features by generation-time correlation and steer (CorrSteer-A) | Gemma-2 2B · MMLU | accuracy: 52.21 -> 55.48 | Improved |
| add validation-based pruning of features (CorrSteer-P over CorrSteer-A) (ablation) | CorrSteer · BBQ Ambig | accuracy: 62.06 -> 66.00 | Improved |
| use many layer-wise features (CorrSteer-A) instead of a single feature (CorrSteer-S) (ablation) | CorrSteer · Eight benchmarks, Gemma-2 2B | accuracy: A beats S on 5 of 8 tasks | Mixed |
| use max-pooling instead of mean-pooling for feature aggregation (ablation) | CorrSteer · HarmBench | refusal rate: 0.00 -> 67.50 | Improved |
| use max-pooling instead of all-token pooling (ablation) | CorrSteer · MMLU | accuracy: 52.91 -> 56.32 | Improved |
| steer with positively correlated features only, versus negatively correlated features across layers (ablation) | CorrSteer · BBQ Disambig | accuracy: 12.15 -> 76.53 | Improved |
| negatively correlated single feature (Neg-S) steering (ablation) | CorrSteer · MMLU | accuracy: 52.23 -> 52.24 | No clear effect |
| correlation-based selection instead of random feature selection (ablation) | CorrSteer-A · MMLU | accuracy: 6.29 -> 55.48 | Improved |
| use true correctness labels instead of permuted labels (ablation) | CorrSteer-A · MMLU | accuracy: 6.24 -> 55.48 | Improved |
| keep only semantic features, remove structural/formatting features (ablation) | CorrSteer-A · BBQ Ambig | accuracy: 62.06 -> 63.93 | Improved |
| keep only semantic features on MMLU (ablation) | CorrSteer-A · MMLU | accuracy: 55.48 -> 55.12 | No clear effect |
| scale steering coefficient from 1.0x to 2.0x (ablation) | CorrSteer on HarmBench · HarmBench, Gemma-2 2B seed 42 | refusal rate: 60.36% -> 7.50% | Hurt |
Closest papers by meaning
- Focus On This, Not That! Steering LLMs with Adaptive Feature Specification (ICML 2025 · similarity 0.88)
- AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoders (ICML 2025 · similarity 0.87)
- Beyond Linear Activation Steering: Invertible Latent Transformations for Controlling LLM Behavior (NeurIPS 2026 · similarity 0.87)
- When Is Rank-1 Steering Cheap? Geometry, Granularity, and Budgeted Search (NeurIPS 2026 · similarity 0.87)
- ADAPT: Hybrid Prompt Optimization for LLM Feature Visualization (NeurIPS 2026 · similarity 0.87)
- Graph-Regularized Sparse Autoencoders for LLM Safety Steering (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
