Research map / Label-efficient and robust learning / OOD and anomaly detection
Kernel PCA for Out-of-Distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CoP and CoRP, kernel PCA detectors for OoD detection that apply a cosine feature mapping (and an RFF-approximated Gaussian on top) before PCA reconstruction error, achieving SOTA with O(1)/O(M) inference cost.
Problem
PCA reconstruction error on penultimate DNN features performs poorly for OoD detection because InD and OoD features are not linearly separable, and kernel PCA with kernel matrices is too costly for large training sets.
Approach
Derives a cosine kernel from the l2-normalization used in KNN detection and a cosine-Gaussian kernel (Gaussian on top of cosine features, approximated with random Fourier features), then runs covariance-based PCA in the explicit mapped feature space so reconstruction errors are computed without a kernel matrix.
Key results
- On ImageNet-1K with ResNet50 under standard training, CoRP reaches average FPR95 48.60 and AUROC 87.53, outperforming KNN (53.97 / 85.01).
- With supervised contrastive learning on ImageNet-1K, CoRP obtains average FPR95 32.11 and AUROC 92.86 versus KNN's 38.47 / 90.91.
- Fused with ReAct on ImageNet-1K ResNet50, CoRP achieves average FPR95 17.68 and AUROC 95.98, reported as new SOTA among compared detectors.
- CoRP inference takes about 0.086 ms per sample versus about 15.59 ms for KNN, with storage about 29 MiB versus about 20 GiB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the cosine-Gaussian mapping instead of using KNN | OoD detection on ResNet50 · ImageNet-1K, standard training | FPR: 53.97 -> 48.60 | Helped |
| add the cosine-Gaussian mapping instead of using KNN | OoD detection on ResNet50 · ImageNet-1K, supervised contrastive learning | AUROC: 90.91 -> 92.86 | Helped |
| add the cosine-Gaussian mapping instead of using KNN | OoD detection on ResNet18 · CIFAR10, standard training | FPR: 30.77 -> 27.34 | Helped |
| add the cosine-Gaussian mapping instead of using KNN | OoD detection on ResNet18 · CIFAR10, supervised contrastive learning | AUROC: 97.93 -> 98.37 | Helped |
| add the Gaussian mapping on top of the cosine mapping (ablation) | CoP / CoRP · ImageNet-1K, standard training (ResNet50), average OoD sets | FPR: 58.40 -> 48.60 | Helped |
| add the Gaussian mapping on top of the cosine mapping (ablation) | CoP / CoRP · ImageNet-1K, standard training (ResNet50), average OoD sets | AUROC: 83.86 -> 87.53 | Helped |
| add the cosine mapping to Gaussian KPCA (ablation) | CoRP · ImageNet-1K, standard training (ResNet50), average OoD sets | FPR: 94.88 -> 48.60 | Helped |
| add the cosine mapping to Gaussian KPCA (ablation) | CoRP · ImageNet-1K, standard training (ResNet50), average OoD sets | AUROC: 50.60 -> 87.53 | Helped |
| replace cosine-Gaussian mapping with standard PCA in the original feature space (ablation) | CoRP · ImageNet-1K, standard training (ResNet50), average OoD sets | FPR: 48.60 -> 84.41 | Hurt |
| replace the Gaussian mapping with a polynomial mapping (ablation) | CoRP · ImageNet-1K, standard training (ResNet50), average OoD sets | FPR: 48.60 -> 68.01 | Hurt |
| replace the Gaussian mapping with a Laplacian mapping (ablation) | CoRP · ImageNet-1K, standard training (ResNet50), average OoD sets | FPR: 48.60 -> 62.54 | Hurt |
Surprising findings
- A Gaussian kernel alone on features yields near-complete OoD detection failure (about 95% FPR), so the cosine normalization is essential.
Limitations
- The two kernels are manually selected with carefully tuned hyper-parameters.
- Kernel parameters are not learned from data; deep kernel learning is suggested as future work.
Experimental setup
Datasets: CIFAR10 ImageNet-1K
Benchmarks: CIFAR10 ImageNet-1K SVHN LSUN iSUN Textures Places365 iNaturalist SUN
Metrics: FPR95 AUROC
Baselines: MSP ODIN Energy GODIN Mahalanobis KNN SSD CSI ReAct BATS ViM DICE
Builds on: PCA reconstruction error OoD detection KNN OoD detection random Fourier features kernel PCA
Closest papers by meaning
- Out-of-Distribution Detection via Conditional Kernel Independence Model (NeurIPS 2022 · similarity 0.89)
- GradPCA: Leveraging NTK Alignment for Reliable Out-of-Distribution Detection (ICLR 2026 · similarity 0.89)
- A Statistical Framework for Efficient Out of Distribution Detection in Deep Neural Networks (ICLR 2022 · similarity 0.88)
- Exploiting Discrepancy in Feature Statistic for Out-of-Distribution Detection (AAAI 2024 · similarity 0.86)
- Learning to Shape In-distribution Feature Space for Out-of-distribution Detection (NeurIPS 2024 · similarity 0.86)
- Test-Time Linear Out-of-Distribution Detection (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.