Research map / Label-efficient and robust learning / Recognition and long-tail
LEACE: Perfect linear concept erasure in closed form
Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, Stella Biderman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LEACE instead of INLP/RLACE for gender erasure | BERT [CLS] representation · Biographies gender erasure | gender prediction accuracy vs edit size | Improved |
| apply LEACE to BERT representations | profession classifier · Biographies | TPR gap RMS: 0.198 -> 0.084 | Improved |
| apply LEACE to BERT representations | profession classifier · Biographies | profession accuracy: 79.3 -> 77.3 | Hurt |
| apply LEACE to BERT representations | profession classifier · Biographies | correlation of TPR gap with % women: 0.867 -> 0.392 | Improved |
| refit profession classifier on projected representations (ablation) | LEACE · Biographies | profession accuracy and TPR gap: 77.3 -> 78.1 acc; 0.084 -> 0.158 TPR gap | Mixed |
| use LEACE instead of SAL for concept scrubbing (ablation) | LLaMA/Pythia · POS scrubbing | perplexity increase (lower is less damage): LLaMA 7B 3.24 -> 1.73 | Improved |
| erase POS with LEACE vs random same-rank subspace (ablation) | LLaMA 7B · POS scrubbing | perplexity (bits per byte): 0.69 -> 1.73 | Hurt |
| LEACE instead of INLP for erasing POS from BERT layer | BERT amnesic probing · POS erasure | rank reduction: 360 -> 17 | Improved |
Later papers that compared with this method
4 later accepted papers report LEACE as a baseline in their results tables; 3 of them report a better number on most of the results they share with it.
- From Measurement to Mitigation: Quantifying and Reducing Identity Leakage in Image Representation Encoders with Linear Subspace Removal (CVPR 2026 · better on 6 of 6 shared results)
- GRADIEND: Feature Learning within Neural Networks Exemplified through Biases (ICLR 2026 · better on 1 of 1 shared result)
- Preserving Task-Relevant Information Under Linear Concept Removal (NeurIPS 2025 · better on 6 of 6 shared results)
- Removing Spurious Concepts from Neural Network Representations via Joint Subspace Estimation (ICML 2024 · better on 0 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Erasing Conceptual Knowledge from Language Models (NeurIPS 2025 · similarity 0.84)
- Localized Concept Erasure for Text-to-Image Diffusion Models Using Training-Free Gated Low-Rank Adaptation (CVPR 2025 · similarity 0.84)
- Minimalist Concept Erasure in Generative Models (ICML 2025 · similarity 0.84)
- Obliviator Reveals the Cost of Nonlinear Guardedness in Concept Erasure (NeurIPS 2025 · similarity 0.84)
- Preserving Task-Relevant Information Under Linear Concept Removal (NeurIPS 2025 · similarity 0.84)
- EraseAnything: Enabling Concept Erasure in Rectified Flow Transformers (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
