Research map / Large language models / Language, safety and interpretability
Representation Surgery: Theory and Practice of Affine Steering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives least-squares-optimal affine steering functions (mean matching and mean+covariance matching, the latter linked to optimal transport between Gaussians) and applies them to reduce gender/dialect bias in classification and toxicity in GPT-2 generation.
Problem
Steering language model representations to suppress undesirable behavior (gender bias, dialect bias, toxicity) while changing representations minimally; prior steering vectors and concept erasure lacked formal justification for steering.
Approach
Shows via affine guardedness and least-squares optimality that steering only requires matching concept-conditional means (justifying steering vectors), and derives a mean+covariance matching affine map (MiMiC) that also matches second moments, is optimal in the OT/Gaussian sense, and provably eliminates expected bias by neighbors.
Key results
- On Bios, mean+covariance matching reduces RMS TPR gap to 0.093 (BERT-base, from 0.155), 0.070 (GPT-2, from 0.168) and 0.085 (Llama2-7b, from 0.143), outperforming LEACE and post-processing.
- On RealToxicityPrompts with GPT-2 large, mean+covariance matching lowers expected max toxicity from 0.39 to 0.29 (about 25% reduction) and toxicity probability from 0.25 to 0.09, while mean matching reaches 0.33.
- Applying mean+covariance matching to all vectors reaches expected max toxicity 0.17 but raises WikiText-2 perplexity from 22.6 to 54.0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply mean and covariance matching | BERT-base profession classifier · Bios | TPR RMS: 0.155 -> 0.093 | Helped |
| Apply mean and covariance matching | GPT-2 profession classifier · Bios | TPR RMS: 0.168 -> 0.070 | Helped |
| Apply mean and covariance matching | Llama2-7b profession classifier · Bios | TPR RMS: 0.143 -> 0.085 | Helped |
| Apply mean and covariance matching | BERT-base profession classifier · Bios | Accuracy: 0.799 -> 0.785 | Hurt |
| Apply mean and covariance matching rather than mean matching alone | GPT-2 profession classifier · Bios | TPR RMS: 0.094 -> 0.070 | Helped |
| Apply mean matching | GPT-2 (large) generation · Real Toxicity Prompts | Expected maximum toxicity: 0.39 -> 0.33 | Helped |
| Apply mean and covariance matching | GPT-2 (large) generation · Real Toxicity Prompts | Expected maximum toxicity: 0.39 -> 0.29 | Helped |
| Apply mean and covariance matching rather than mean matching alone | GPT-2 (large) generation · Real Toxicity Prompts | Expected maximum toxicity: 0.33 -> 0.29 | Helped |
| Apply mean and covariance matching | Llama2-7b sentiment classifier · Dialect bias (AAE 0.950) | TPR gap: 0.193 -> 0.086 | Helped |
| Apply mean and covariance matching to all vectors instead of selectively (ablation) | GPT-2 (large) generation · Real Toxicity Prompts | Expected maximum toxicity: 0.29 -> 0.17 | Helped |
| Apply mean and covariance matching to all vectors instead of selectively (ablation) | GPT-2 (large) representations · WikiText-2 | Perplexity: 24.2 -> 54.0 | Hurt |
| Apply mean matching to all vectors instead of selectively (ablation) | GPT-2 (large) generation · Real Toxicity Prompts | Expected maximum toxicity: 0.33 -> 0.28 | Helped |
Surprising findings
- Applying the covariance-matching steering to all vectors (not only source class) gives the strongest toxicity reduction but severely damages perplexity on WikiText-2.
- Steering with mean matching alone is roughly as effective as LEACE for gender bias, and the covariance-matching variant is the one that clearly reduces bias by neighbors to the random baseline (~52%).
Limitations
- Affine (linear) transformations may not reach state-of-the-art toxicity mitigation.
- Distribution mismatch: maps fit on last-token representations but applied at every generation step.
- Concept function must be approximated by an MLP classifier, and covariance-matching to all vectors harms fluency/perplexity.
Experimental setup
Datasets: Bios Twitter AAE/SAE dialect dataset Jigsaw Toxic Comments Classification Challenge RealToxicityPrompts WikiText-2
Benchmarks: Bios RealToxicityPrompts WikiText-2
Metrics: RMS TPR gap accuracy expected maximum toxicity toxicity probability fluency (GPT-2 XL perplexity) distinct n-gram ratio WikiText-2 perplexity bias by neighbors (k-NN gender agreement)
Baselines: LEACE Postprocessing (Wang et al.) DAPT GeDI PPLM UDDIA DExperts GOODTRIEVER
Builds on: LEACE steering vectors affine guardedness bias by neighbors
Closest papers by meaning
- Improved Representation Steering for Language Models (NeurIPS 2025 · similarity 0.89)
- To Steer or Not to Steer? Mechanistic Error Reduction with Abstention for Language Models (ICML 2025 · similarity 0.87)
- Personalized Steering of Large Language Models: Versatile Steering Vectors Through Bi-directional Preference Optimization (NeurIPS 2024 · similarity 0.87)
- Steering When Necessary: Flexible Steering Large Language Models with Backtracking (NeurIPS 2025 · similarity 0.87)
- Analysing the Generalisation and Reliability of Steering Vectors (NeurIPS 2024 · similarity 0.87)
- The Sharp Directions Are Against You: Curvature Analysis of Activation Steering (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.