Research map / Large language models / Language, safety and interpretability

Mitigating Memorization Where It Happens

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: LLM pruning and scaling data

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Gated Subspace Steering (GSS), an inference-time activation steering method that uses a target-reference token-level log-likelihood signal to calibrate low-rank probe-steer directions and applies corrections only when a probe gate fires, reducing verbatim memorization with less utility loss.

Problem

LLMs memorize and reproduce training sequences verbatim, but existing mitigation methods apply interventions uniformly, degrading utility on the majority of tokens that generalize normally.

Approach

Use the token-wise target-minus-reference log-likelihood difference to localize high-signal positions, derive closed-form probe-steer directions via SVD of a gradient-weighted activation matrix (whitened or contrastive), and apply activation corrections only through a threshold gate at inference without updating weights or querying the reference.

Key results

• On Pythia-6.9B, GSS reduces (n,k)-memorization from 89.31% to 6.96% with test perplexity rising from 19.46 to 29.15.
• On TinyMem, GSS achieves 0.00% memorization across math and language models under noise and backdoor settings, with inference overhead around 0.001 seconds.
• Gated intervention yields 2.4–4.1 lower perplexity than the ungated variant while both reach 0.0% memorization on TinyMem.

Surprising findings

• Target-reference signal ranks tokens worse than target confidence in token-level verbatim continuation (AUROC 0.583 vs 0.909), and does not rank the most verbatim-relevant positions within memorized sequences.
• As the reference becomes more exposed to the evaluated sequences, the localization signal collapses (sequence AUROC falls from 0.92 to 0.49).

Limitations

• Perplexity cost is substantial on pretrained Pythia models (+29.9% to +49.8%), since pretrained models lack a clean forget/retain split.
• The closed-form optimality applies only to a local linear surrogate; sequence-level suppression and utility are evaluated empirically.
• Threshold exceedance rates shift under distribution shift (e.g., 0.038 to 0.127 vs nominal 0.1).

Experimental setup

Datasets: TinyMem The Pile GSM8K UltraChat

Benchmarks: TinyMem Pythia pretraining corpus (The Pile) GSM8K UltraChat

Metrics: (n,k)-memorization perplexity accuracy wall-clock time

Baselines: GA NPO Greedy SOU Dura Subnet BalSub ROME MEMIT TaskVector LEACE AlphaSteer

Builds on: contrastive activation addition LEACE NPO exposure metric

Closest papers by meaning

• Mitigating Memorization in Language Models (ICLR 2025 · similarity 0.91)
• Memorization Sinks: Isolating Memorization during LLM Training (ICML 2025 · similarity 0.89)
• Learning the Signature of Memorization in Autoregressive Language Models (NeurIPS 2026 · similarity 0.89)
• Memory Inception: Latent-Space KV Cache Manipulation for Steering LLMs (NeurIPS 2026 · similarity 0.88)
• Characterizing Memorization in Diffusion Language Models: Generalized Extraction and Sampling Effects (NeurIPS 2026 · similarity 0.87)
• Rethinking LLM Memorization through the Lens of Adversarial Compression (NeurIPS 2024 · similarity 0.87)