Research map / Label-efficient and robust learning / OOD and anomaly detection

Local Sparsity Enables Unsupervised LLM Safety Detection

NeurIPS 2026 · Label-efficient and robust learning › OOD and anomaly detection · approach: Deep one-class and tabular anomaly detection · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes one-class LLM safety detection using local sparsity of sparse autoencoder features (FreqMask-KM and LearnedMask-LoRA), with a sample-complexity bound and cheap per-cluster calibration from 1% labeled data.

Problem

Deployment-time LLM safety detection is usually supervised and needs labeled unsafe data; unsupervised anomaly detection on high-dimensional LLM activations is assumed to perform poorly.

Approach

Under the linear representation hypothesis, SAE feature supports are shared across local neighborhoods; clustering safe SAE features and scoring with a per-cluster local frequency mask (or per-cluster LoRA adapters) yields a one-class detector whose effective dimension is the local sparsity, plus a per-cluster affine calibration from a small labeled budget.

Key results

• FreqMask-KM reaches 0.937 AUROC on LLaMA3-8B and LearnedMask-LoRA reaches 0.904 on Gemma-4-26B, the best unsupervised results, versus 0.972 for the supervised linear probe on LLaMA3-8B.
• Mean AUROC over the five >=8B models: LearnedMask-LoRA 0.909 and FreqMask-KM 0.904, vs GMM 0.903 and Mahalanobis 0.893; CVDD 0.692.
• With 1% unsafe and 1% benign OOD calibration data, AUROC rises from 0.265-0.750 to 0.962-0.990 across six models.

Surprising findings

• Substituting the SAE encoder into CVDD raises its mean AUROC from 0.692 to 0.806, showing representation space matters more than the scoring method.
• Per-cluster top-k masking performed worse (0.790) than global frequency masking (0.904) in the decomposition study.

Limitations

• Gap of roughly 0.04-0.08 AUROC to the supervised linear probe on models of 8B and larger.
• Performance on Qwen2-1.5B is weaker (all unsupervised methods below 0.80 AUROC), so representation quality bounds detection.
• Unsupervised detector is sensitive to benign distribution shift, requiring small labeled calibration to fix false positives.

Experimental setup

Datasets: BeaverTails ToxiGen HarmBench HellaSwag HH-RLHF-helpful

Benchmarks: BeaverTails ToxiGen HarmBench HellaSwag HH-RLHF-helpful

Metrics: AUROC TPR@5%FPR false positive rate on benign OOD

Baselines: Mahalanobis distance GMM One-Class SVM Deep SVDD CVDD supervised linear probe (BCE)

Builds on: linear representation hypothesis sparse autoencoders CVDD Deep SVDD LoRA

Closest papers by meaning

• Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge (NeurIPS 2026 · similarity 0.88)
• Uncovering Safety Risks of Large Language Models through Concept Activation Vector (NeurIPS 2024 · similarity 0.88)
• Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models (NeurIPS 2024 · similarity 0.87)
• Safety Layers in Aligned Large Language Models: The Key to LLM Security (ICLR 2025 · similarity 0.87)
• Certifying Language Model Robustness with Fuzzed Randomized Smoothing: An Efficient Defense Against Backdoor Attacks (ICLR 2025 · similarity 0.87)
• Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space (NeurIPS 2024 · similarity 0.87)