Research map / Label-efficient and robust learning / OOD and anomaly detection
Language-Assisted Feature Transformation for Anomaly Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LAFT, a training-free method that uses CLIP text-derived concept axes (PCA over pairwise prompt differences) to project image features toward or away from user-specified attributes, combined with kNN for semantic anomaly detection and with WinCLIP for industrial anomaly detection.
Problem
Anomaly detection models trained only on normal data define normality from biased or limited data and cannot easily let users specify which attributes matter or should be ignored, especially when attributes are entangled or nuisance attributes are present.
Approach
Build concept subspaces from CLIP text embeddings of concept values (pairwise differences reduced by PCA) and transform image features by projecting onto (guide) or orthogonally away from (ignore) these axes, with no additional training, then score anomalies with kNN.
Key results
- On Colored MNIST (number guidance), LAFT AD reaches 98.5% AUROC versus 92.4% for kNN with a normal subset and 91.2% for ZOE.
- On CelebA Eyeglasses, LAFT AD achieves 98.1% AUROC and 5.9% FPR95, compared with 87.8% AUROC for InCTRL.
- WinCLIP+LAFT consistently outperforms WinCLIP in zero-shot and few-shot industrial settings and matches or beats InCTRL without additional adapter training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply LAFT to guide number anomalies | LAFT AD vs. kNN using a normal-image subset · Colored MNIST: Number | AUROC: 92.4 -> 98.5 | Helped |
| Apply LAFT to guide bird anomalies | LAFT AD vs. kNN using a normal-image subset · Waterbirds: Bird | AUROC: 82.3 -> 95.6 | Helped |
| Apply LAFT to guide hair-color anomalies | LAFT AD vs. kNN using a normal-image subset · CelebA: Hair color | AUROC: 83.3 -> 95.0 | Helped |
| Apply LAFT to guide eyeglasses anomalies | LAFT AD vs. kNN using a normal-image subset · CelebA: Eyeglasses | AUROC: 83.0 -> 98.1 | Helped |
| Add LAFT General to WinCLIP | WinCLIP · MVTec AD few-shot (K=0) | AUROC: 90.4 -> 92.6 | Helped |
| Add LAFT Category to WinCLIP | WinCLIP · VisA few-shot (K=0) | AUROC: 75.5 -> 80.6 | Helped |
| Add auxiliary concept values to the prompts (ablation) | LAFT AD (all candidate prompt variant) · Colored MNIST: Number | AUROC: 98.8 -> 98.5 | No clear effect |
| Add auxiliary concept values to the prompts (ablation) | LAFT AD (all candidate prompt variant) · Waterbirds: Bird | AUROC: 95.9 -> 95.6 | No clear effect |
| Set guide PCA components d from 2 to 24 (ablation) | LAFT AD · Colored MNIST: Number | AUROC: 80.3 -> 98.5 | Helped |
| Set guide PCA components d from 18 to 64 (ablation) | LAFT AD · Colored MNIST: Number | AUROC: 98.2 -> 97.1 | Hurt |
| Set guide PCA components d from 2 to 18 (ablation) | LAFT AD · Waterbirds: Bird | AUROC: 87.2 -> 95.6 | Helped |
| Set ignore PCA components d from 8 to 224 (ablation) | LAFT AD · Colored MNIST: Number | AUROC: 96.2 -> 97.4 | Helped |
Surprising findings
- Language-only guided methods (MCM, CLIPN) can underperform a plain image kNN baseline because inaccurate prompts for anomalous values mislead image-text similarity.
- Ignoring an attribute with LAFT does not directly improve performance on the other attribute in real-world datasets such as Waterbirds, though the ignored attribute is suppressed.
Limitations
- Choice of PCA component count d is heuristic and sensitive, with no automatic selection in unsupervised AD.
- Ignoring real-world attributes via text prompts is hard; removing all attribute information is incomplete, so suppression is partial.
- Requires user knowledge of attribute values; performance depends on prompt quality.
Experimental setup
Datasets: Colored MNIST Waterbirds CelebA MVTec AD VisA
Benchmarks: Colored MNIST Waterbirds CelebA MVTec AD VisA
Metrics: AUROC AUPRC FPR95
Baselines: kNN LinearProbe MCM ZOE CLIPN WinCLIP WinCLIP+ InCTRL
Builds on: CLIP WinCLIP kNN anomaly scoring MCM ZOE
Closest papers by meaning
- AnomalyGPT: Detecting Industrial Anomalies Using Large Vision-Language Models (AAAI 2024 · similarity 0.87)
- Adapting Visual-Language Models for Generalizable Anomaly Detection in Medical Images (CVPR 2024 · similarity 0.86)
- MetaUAS: Universal Anomaly Segmentation with One-Prompt Meta-Learning (NeurIPS 2024 · similarity 0.86)
- One-to-Normal: Anomaly Personalization for Few-shot Anomaly Detection (NeurIPS 2024 · similarity 0.85)
- LogicAD: Explainable Anomaly Detection via VLM-based Text Feature Extraction (AAAI 2025 · similarity 0.85)
- Harnessing Large Language Models for Training-free Video Anomaly Detection (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.