Research map / Large language models / Language, safety and interpretability

Separate the Wheat from the Chaff: Model Deficiency Unlearning via Parameter-Efficient Module Operation

AAAI 2024 · Large language models › Language, safety and interpretability · approach: LLM pruning and scaling data · 10 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Extraction-before-Subtraction (Ext-Sub), which extracts the deficiency component of an anti-expert LoRA via vector geometry before subtracting it from an expert LoRA, to unlearn untruthfulness and toxicity in LLaMA-7B.

Problem

Direct subtraction of anti-expert parameter-efficient modules (PEMs) to unlearn untruthfulness and toxicity in LLMs causes harmful forgetting, text degeneration and loss of general capabilities.

Approach

Decompose each row of the anti-expert LoRA update into a general component (projection onto the direction of the sum of unit expert and anti-expert row vectors) and a residual deficiency component, then subtract only the deficiency component from the expert LoRA.

Key results

• On TruthfulQA with LLaMA-7B (Alpaca-GPT4), Ext-Sub at lambda=2.0 reaches mc1 36.0 and mc2 55.2 vs expert 33.3/52.8 and direct subtraction best 33.5/52.7.
• On toxicity, Ext-Sub (alpaca expert, lambda=2.0) lowers toxic response ratio from 12.5% (expert) to 6.0%, versus 10.0% for direct subtraction.
• Ext-Sub preserves fundamental abilities roughly at baseline (e.g., average 24.6 vs 25.1 expert across MMLU/GSM/BBH) while direct subtraction avoids degradation only at small lambda and produces repetition for lambda > 0.6.

Measured effects reported in the paper

Surprising findings

• Anti-expert PEMs trained on untruthful data still retain useful language modeling and narrative capabilities, motivating extraction rather than full negation.
• Unlearning order matters substantially for compositional detoxification and truthfulness unlearning.

Limitations

• Operating on the full LoRA matrix yields high-rank updates that cannot be stored as low-rank matrices, increasing storage cost.
• Evaluated only on LLaMA-7B/OPT-6.7B/LLaMA-13B with LoRA (and prefix-tuning in appendix); generalization to other scales and PEM types is unverified.
• Ext-Sub underperforms direct subtraction on HaluEval Summary and some free-generation metrics; toxicity evaluation via Detoxify has acknowledged flaws.

Experimental setup

Datasets: Alpaca-GPT4 WizardLM-70k TruthfulQA HaluEval Civil Comments AlpacaEval MMLU GSM8K BBH

Benchmarks: TruthfulQA HaluEval Toxicity test set (200 instructions) MMLU GSM BBH AlpacaEval next-token accuracy on AlpacaEval test data

Metrics: mc1 mc2 bleu acc rouge1 acc true % true&info % QA accuracy Summary accuracy Detoxify toxicity score toxic response ratio %

Baselines: direct subtraction (theta+ minus lambda theta-) expert PEM anti-expert PEM

Builds on: LoRA direct PEM subtraction (negative/anti-expert modules)

Closest papers by meaning

• Unified Parameter-Efficient Unlearning for LLMs (ICLR 2025 · similarity 0.90)
• Towards Effective Evaluations and Comparisons for LLM Unlearning Methods (ICLR 2025 · similarity 0.90)
• A Closer Look at Machine Unlearning for Large Language Models (ICLR 2025 · similarity 0.90)
• Catastrophic Failure of LLM Unlearning via Quantization (ICLR 2025 · similarity 0.89)
• In-Context Unlearning: Language Models as Few-Shot Unlearners (ICML 2024 · similarity 0.89)
• MUSE: Machine Unlearning Six-Way Evaluation for Language Models (ICLR 2025 · similarity 0.89)