Research map / Label-efficient and robust learning / OOD and anomaly detection
SeTAR: Out-of-Distribution Detection with Selective Low-Rank Approximation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SeTAR, a training-free OOD detector that applies post-hoc selective low-rank approximation (discarding minor singular components) to CLIP W_up matrices via a greedy top-to-bottom search, plus SeTAR+FT, which freezes the principal components and fine-tunes the minor ones.
Problem
OOD detection with CLIP: zero-shot scoring methods suffer from domain gaps, fine-tuning methods risk damaging pretrained representations, and sparsification methods designed for CNNs weaken on large pretrained models like CLIP.
Approach
Selectively replace weight matrices (mainly W_up) with low-rank SVD approximations that keep principal components and drop minor ones, with the rank reduction per layer chosen by a greedy search over a validation set using the LoCoOp loss; the fine-tuning extension freezes the principal part and updates only the minor components.
Key results
- On Pascal-VOC with CLIP-base, SeTAR+GL-MCM reduces average FPR95 from 29.44 to 23.86 (about 18.95% relative) versus vanilla GL-MCM.
- On ImageNet1K with CLIP-base, SeTAR+GL-MCM reaches 91.32% average AUROC; SeTAR+FT with GL-MCM reaches 92.31% AUROC.
- SeTAR+FT lowers average FPR95 with CLIP-base on ImageNet1K (38.77 vs 41.67 for LoRA under MCM), and on Swin-base with Energy score cuts FPR95 from 62.17 (LoRA) to 39.29.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply selective low-rank approximation to CLIP weights before MCM scoring | CLIP-base · ImageNet1K OOD average | FPR95: 43.09 -> 40.24 | Helped |
| Apply selective low-rank approximation to CLIP weights before GL-MCM scoring | CLIP-base · Pascal-VOC OOD average | FPR95: 29.44 -> 23.86 | Helped |
| Use SeTAR initialization for fine-tuning instead of LoCoOp | CLIP-base with MCM · ImageNet1K OOD average | FPR95: 39.76 -> 38.77 | Helped |
| Modify vision and text modalities rather than vision only (ablation) | SeTAR with MCM · Pascal-VOC OOD average | FPR95: 33.19 -> 32.46 | Helped |
| Modify vision modality rather than text modality only (ablation) | SeTAR with MCM · ImageNet1K OOD average | FPR95: 42.78 -> 40.27 | Helped |
| Leave projection matrix W_p unmodified (ablation) | SeTAR with GL-MCM · ImageNet1K OOD average | FPR95: 34.30 -> 33.12 | Helped |
| Use sequential top-to-bottom search instead of layer-exhaustive search (ablation) | SeTAR with MCM · ImageNet1K OOD average | FPR95: 41.99 -> 40.24 | Helped |
| Prune minor rather than principal singular components (ablation) | SeTAR with MCM · ImageNet1K OOD average | FPR95: 43.09 -> 40.24 | Helped |
| Prune minor rather than random singular components (ablation) | SeTAR with MCM · Pascal-VOC OOD average | AUROC: 93.09 -> 93.74 | Helped |
Surprising findings
- Principal-component pruning (keeping minor components) hurts sharply, and random pruning performs better than principal pruning, contrary to the usual intuition about SVD truncation.
- The benefit of GL-MCM over MCM depends on backbone: with CLIP-large, SeTAR+MCM beats SeTAR+GL-MCM on ImageNet1K.
Limitations
- Optimal hyperparameters (lambda, top-K) vary across backbones and must be set per dataset and model.
- SeTAR+FT is a simple fine-tuning design and may not be the most effective adaptation strategy.
- Evaluated only on visual OOD inputs; other modalities such as text, audio and video are not explored.
Experimental setup
Datasets: ImageNet1K Pascal-VOC iNaturalist SUN Places Texture ImageNet22K COCO SSB-Hard
Benchmarks: ImageNet1K OOD benchmark Pascal-VOC OOD benchmark SSB-Hard near-OOD
Metrics: FPR95 AUROC
Baselines: MCM GL-MCM NPOS CoOp LoCoOp LoRA NegLabel ReAct ASH DICE Energy Softmax
Builds on: MCM GL-MCM LoCoOp LoRA LASER
Closest papers by meaning
- Leveraging Perturbation Robustness to Enhance Out-of-Distribution Detection (CVPR 2025 · similarity 0.89)
- A Systematic Analysis of Out-of-Distribution Detection Under Representation and Training Paradigm Shifts (NeurIPS 2026 · similarity 0.88)
- Image Background Serves as Good Proxy for Out-of-distribution Data (ICLR 2024 · similarity 0.87)
- RUNA: Object-Level Out-of-Distribution Detection via Regional Uncertainty Alignment of Multimodal Representations (AAAI 2025 · similarity 0.87)
- Adaptive Multi-prompt Contrastive Network for Few-shot Out-of-distribution Detection (ICML 2025 · similarity 0.87)
- Detecting Out-of-Distribution Through the Lens of Neural Collapse (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.