Research map / Label-efficient and robust learning / Recognition and long-tail
Unlocking Better Closed-Set Alignment Based on Neural Collapse for Open-Set Recognition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Dual ETF (DEF) loss with a Fixed ETF Template to force closed-set class means, features, and classifier vectors onto a simplex ETF (neural collapse), improving open-set recognition on standard OSR benchmarks.
Problem
Open-set recognition relies on closed-set feature discriminability, but it is unclear how DNNs achieve discriminative closed-set boundaries; existing methods only partially satisfy neural collapse (ETF alignment and self-duality).
Approach
Introduces a Fixed ETF Template (FiT) as an ideal fixed simplex ETF for C closed-set classes, and a Dual ETF loss with F-DEF (aligns class means/features to FiT in feature space via SupCon-trained encoder) and C-DEF (aligns classifier vectors to FiT for self-duality with class means) to strictly satisfy NC2 and NC3.
Key results
- DEF achieves 95.5% average AUROC across five OSR benchmarks, versus 95.0% for DCTAU (Table 1).
- On TinyImageNet, DEF reaches 85.6% AUROC, a 2.0-point gain over DCTAU's 83.6%.
- DEF trains faster than DCTAU; e.g., on CIFAR10 it takes 1.51 vs 4.61 hours for 600 epochs (Table 3).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DEF instead of DCTAU | OSR · SVHN | AUROC (%): 99.2 -> 99.1 | Hurt |
| Use DEF instead of DCTAU | OSR · CIFAR10 | AUROC (%): 95.6 -> 95.7 | Helped |
| Use DEF instead of DCTAU | OSR · CIFAR+10 | AUROC (%): 98.5 -> 98.9 | Helped |
| Use DEF instead of DCTAU | OSR · CIFAR+50 | AUROC (%): 98.1 -> 98.4 | Helped |
| Use DEF instead of DCTAU | OSR · TinyImageNet | AUROC (%): 83.6 -> 85.6 | Helped |
| Add F-DEF loss to DEF (ablation) | DEF · CIFAR10 | AUROC: 94.03 -> 95.69 | Helped |
| Add F-DEF loss to DEF (ablation) | DEF · TinyImageNet | AUROC: 81.72 -> 85.60 | Helped |
| Add C-DEF loss to DEF (ablation) | DEF · CIFAR10 | AUROC: 95.55 -> 95.69 | Helped |
| Add C-DEF loss to DEF (ablation) | DEF · TinyImageNet | AUROC: 84.95 -> 85.60 | Helped |
| Add F-DEF loss to DEF (ablation) | DEF · CIFAR10 | OSCR: 92.82 -> 94.13 | Helped |
| Add F-DEF loss to DEF (ablation) | DEF · TinyImageNet | OSCR: 77.83 -> 79.76 | Helped |
| Add C-DEF loss to DEF (ablation) | DEF · CIFAR10 | OSCR: 94.09 -> 94.13 | Helped |
Surprising findings
- Existing CE-based OSR methods only partially satisfy neural collapse properties (NC2 and NC3), leaving suboptimal ETF structure.
- Vanilla SupCon achieves 93.4% average AUROC, and DEF's F-DEF component contributes more than C-DEF in ablation on CIFAR10 and TinyImageNet.
Limitations
- Improvements over DCTAU are modest on some datasets (e.g., slightly lower on SVHN).
- Theoretical analysis of open-set misclassification relies on simplifying assumptions (standard normal open-set features).
Experimental setup
Datasets: SVHN CIFAR10 CIFAR+10 CIFAR+50 TinyImageNet MNIST Omniglot ImageNet LSUN
Benchmarks: SVHN CIFAR10 CIFAR+10 CIFAR+50 TinyImageNet MNIST-Noise Omniglot
Metrics: AUROC OSCR OpenAUC macro F1
Baselines: Softmax OpenMax G-OpenMax OSRCI CPN GFROSR PROSER OpenHybrid ARPL Class-inclusion PMAL MLS
Builds on: Neural Collapse supervised contrastive learning MLS (Vaze et al. 2022) DCTAU NECO
Closest papers by meaning
- Guiding Neural Collapse: Optimising Towards the Nearest Simplex Equiangular Tight Frame (NeurIPS 2024 · similarity 0.87)
- Neural Collapse for Cross-entropy Class-Imbalanced Learning with Unconstrained ReLU Features Model (ICML 2024 · similarity 0.87)
- Neural Collapse in Cumulative Link Models for Ordinal Regression: An Analysis with Unconstrained Feature Model (NeurIPS 2025 · similarity 0.87)
- From Coarse to Fine-Grained Open-Set Recognition (CVPR 2024 · similarity 0.86)
- Generalized Neural Collapse for a Large Number of Classes (ICML 2024 · similarity 0.86)
- Wide Neural Networks Trained with Weight Decay Provably Exhibit Neural Collapse (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.