Research map / Large language models / Language, safety and interpretability
Redefining Experts: Interpretable Decomposition of Language Models for Toxicity Mitigation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EigenShift, which SVD-decomposes the LM head and damps singular values of toxicity-aligned "generation expert" directions to reduce toxic generation while preserving perplexity, and introduces a TPH harmonic-mean metric.
Problem
LLM toxic generation must be reduced without damaging fluency; existing neuron-level interventions are unstable, context-dependent, and conflate toxicity detection experts with generation experts, causing catastrophic forgetting and perplexity blow-ups.
Approach
Decompose the final lm_head matrix W = U Σ Vᵀ and dampen singular values of eigen-directions whose activation differs between toxic and non-toxic hidden states, targeting generation-aligned components rather than neurons or early layers, with no retraining.
Key results
- EigenShift reduces toxicity on LLaMA-2-7B by 57.47% (11.13% to 4.71%) with perplexity rising from 6.23 to 9.84, giving a TPH score of 60.37%.
- Layer-wise experts raise average AUROC on Jigsaw from 54.66% (neurons) to 63.32%, while neuron AUROCs cluster near 0.5.
- On MPT-7B, Aura achieves a higher TPH (93.94%) than EigenShift (87.74%), so EigenShift does not win on every model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace neuron-based features with aggregated layer-wise representations | toxicity detection · Jigsaw | AUROC: 54.66 -> 63.32 | Helped |
| replace neuron-based features with aggregated layer-wise representations | toxicity detection · ToxiCN | AUROC: 56.89 -> 60.97 | Helped |
| use EigenShift instead of Aura | LLaMA-2 · RealToxicPrompts | TPH score: 43.73% -> 60.37% | Helped |
| use EigenShift instead of Aura | Mistral-v0.1 · RealToxicPrompts | TPH score: 44.74% -> 57.68% | Helped |
| use EigenShift instead of Aura | GPT-2-xl · RealToxicPrompts | TPH score: 14.66% -> 16.47% | Helped |
| use EigenShift instead of Aura | MPT-7B · RealToxicPrompts | TPH score: 93.94% -> 87.74% | Hurt |
| use EigenShift instead of Aura | Falcon-7B · RealToxicPrompts | TPH score: 77.81% -> 78.86% | Helped |
| set alpha to -1 at Top-k 1024 instead of alpha 0 at Top-k 1024 (ablation) | EigenShift · RealToxicPrompts | TPH score: 60.39% -> 0.00% | Hurt |
| set alpha to 0.9 at Top-k 1024 instead of alpha 0 at Top-k 1024 (ablation) | EigenShift · RealToxicPrompts | TPH score: 60.39% -> 32.48% | Hurt |
| apply EigenShift intervention | LLaMA · MMLU Algebra | accuracy: 35 -> 34 | Mixed |
Surprising findings
- Det-0 and DAMP drive toxicity to near 0% but push perplexity into the thousands, yielding near-zero TPH scores.
- Most neurons have AUROC only slightly above 0.5, so the common AUROC>=0.5 expert criterion is too permissive.
Limitations
- Only the final lm_head is decomposed; early and middle layers are not studied.
- Results are reported on a single toxicity benchmark (RealToxicityPrompts) with perplexity as the fluency proxy.
- Theoretical grounding is claimed but the SVD reconstruction is an approximation of W; the text states it is not a perfect factorization.
Experimental setup
Datasets: Jigsaw ToxiCN RealToxicityPrompts English Wikipedia (20231101 snapshot)
Benchmarks: RealToxicityPrompts Jigsaw ToxiCN MMLU
Metrics: toxicity rate perplexity TPH score AUROC AP F1
Baselines: DetZero DAMP AURA BERT BART
Builds on: expert neurons (Suau et al.) DetZero DAMP AURA
Closest papers by meaning
- Whispering Experts: Neural Interventions for Toxicity Mitigation in Language Models (ICML 2024 · similarity 0.91)
- When Bad Data Leads to Good Models (ICML 2025 · similarity 0.88)
- A Single Neuron Is Sufficient to Bypass Safety Alignment in Large Language Models (NeurIPS 2026 · similarity 0.88)
- Characterizing Large Language Model Geometry Helps Solve Toxicity Detection and Generation (ICML 2024 · similarity 0.88)
- IF-Guide: Influence Function-Guided Detoxification of LLMs (NeurIPS 2025 · similarity 0.88)
- Large Language Models can Become Strong Self-Detoxifiers (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.