Research map / Large language models / Language, safety and interpretability

On Effects of Steering Latent Representation for Large Language Model Unlearning

AAAI 2025 · Large language models › Language, safety and interpretability · approach: Mechanistic analysis of attention and representations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

The paper theoretically and empirically analyzes why Representation Misdirection for Unlearning (RMU) works, and proposes Adaptive RMU, which scales the random target vector by the forget-representation norm so unlearning works across more layers.

Problem

RMU is an effective LLM unlearning method, but its underlying mechanism is poorly understood, and it becomes ineffective when applied to middle and later layers because its fixed steering coefficient c is smaller than forget-representation norms there.

Approach

Provides a theoretical account of RMU (token-confidence/logit variance, noise sensitivity, and gradient-masking view of jailbreak robustness) and replaces the fixed coefficient c with an adaptive one, beta times the norm of the frozen forget representation, so the steering target scales with each input's representation norm.

Key results

• Adaptive RMU (l=7) on Zephyr-7B reaches 23.7% WMDP-Biology, 26.5% WMDP-Cyber and 55.0% MMLU, versus RMU's 28.8%, 28.8% and 56.8%.
• Adaptive RMU reduces average WMDP-Biology accuracy by 15.6% over layers 11-31 where RMU is ineffective, giving an overall 14.3% improvement in drop-in-accuracy on WMDP-Biology.
• Adaptive RMU adds no extra computational cost and needs tuning only beta rather than both c and layer, an L-fold reduction in grid search.

Measured effects reported in the paper

Surprising findings

• Adaptive RMU is almost ineffective for all unlearn layers when the retain set is WMDP-Biology/Cyber instead of Wikitext, attributed to high n-gram overlap between forget and retain sets.
• On the MMLU subset unlearning tasks, Adaptive RMU shows excessive unlearning that substantially degrades the retained MMLU categories.

Limitations

• Beta is still manually tuned via grid search, so the adaptive-coefficient challenge is not fully resolved.
• Analysis and experiments cover only white-box attacks on open-weight models; black-box jailbreaks are not studied.
• Only 7B-class (or equivalent) models are tested, and only three layers are updated per run.

Experimental setup

Datasets: WMDP-Biology forget set WMDP-Cyber forget set Wikitext WMDP Q&A MMLU MMLU subset forget sets (Economics, Law, Physics)

Benchmarks: WMDP-Biology WMDP-Cyber MMLU

Metrics: Q&A accuracy drop-in-accuracy MaxLogit distribution cosine similarity

Baselines: RMU LLMU SCRUB SSD

Builds on: Representation Misdirection for Unlearning (RMU) Greedy Coordinate Gradient (GCG) AdamW

Closest papers by meaning

• Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models (AAAI 2025 · similarity 0.91)
• Large Language Model Unlearning via Embedding-Corrupted Prompts (NeurIPS 2024 · similarity 0.90)
• Keeping an Eye on LLM Unlearning: The Hidden Risk and Remedy (NeurIPS 2025 · similarity 0.90)
• RWKU: Benchmarking Real-World Knowledge Unlearning for Large Language Models (NeurIPS 2024 · similarity 0.90)
• LLM Unlearning via Loss Adjustment with Only Forget Data (ICLR 2025 · similarity 0.90)
• On Large Language Model Continual Unlearning (ICLR 2025 · similarity 0.90)