Research map / Label-efficient and robust learning / Recognition and long-tail
Semantic-Guided Novel Category Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Semantic-guided Novel Category Discovery (SNCD), where unlabeled images come with their class names, and a dual-branch model that jointly clusters and semantically classifies them using dynamic visual prototypes, a GloVe-based semantic similarity matrix, and mutual-information coupling between branches.
Problem
Standard novel category discovery clusters unlabeled images using only a labeled set of disjoint classes and treats class names as one-hot labels, so clusters cannot be semantically named or related. The paper asks how to use the class names of unlabeled images to both cluster and recognize them.
Approach
Combine a visual-prototype classification branch, projected into a semantic space via a word-vector similarity matrix, with a UNO-style clustering branch. Pseudo-labels for clusters come from Hungarian matching of top-K classification anchors, and mutual information between the two branches' predictions couples them.
Key results
- On CIFAR100-20 task-aware, the method reaches 57.8% classification and 93.7% clustering accuracy; clustering is ~4.5% above the best prior NCD method on that split.
- On ImageNet task-aware clustering, the method reports 92.5% versus 91.1% for UNOv2 (about 1% gain).
- Under the task-agnostic protocol on CIFAR100-20 unlabeled, it gains about 9% over the leading method.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add mutual information loss between classification and clustering predictions (ablation) | SNCD · CIFAR100-20 | classification accuracy: 34.0 -> 52.3 | Helped |
| add mutual information loss between classification and clustering predictions (ablation) | SNCD · CIFAR100-20 | clustering accuracy: 92.4 -> 93.0 | Helped |
| add unlabeled-class samples to the visual prototypes (ablation) | SNCD · CIFAR100-20 | classification accuracy: 52.3 -> 57.8 | Helped |
| add unlabeled-class samples to the visual prototypes (ablation) | SNCD · CIFAR100-20 | clustering accuracy: 93.0 -> 93.7 | Helped |
| add the semantic similarity matrix (ablation) | SNCD · CIFAR100-20 | classification accuracy: 50.8 -> 57.8 | Helped |
| add the semantic similarity matrix (ablation) | SNCD · CIFAR100-20 | clustering accuracy: 92.7 -> 93.7 | Helped |
| replace mutual information loss with L2-distance loss (ablation) | SNCD · CIFAR100-20 | classification accuracy: 57.8 -> 30.2 | Hurt |
| replace mutual information loss with L2-distance loss (ablation) | SNCD · CIFAR100-20 | clustering accuracy: 93.7 -> 88.6 | Hurt |
| use semantic-guided novel category discovery instead of UNOv2 | CIFAR100-20 task-aware clustering · CIFAR100-20 | clustering accuracy: 90.2 -> 93.7 | Helped |
| use semantic-guided novel category discovery instead of UNOv2 | ImageNet task-aware clustering · ImageNet | clustering accuracy: 91.1 -> 92.5 | Helped |
| use semantic-guided novel category discovery instead of UNOv2 | CIFAR10 task-aware clustering · CIFAR10 | clustering accuracy: 96.1 -> 93.8 | Hurt |
| use semantic-guided novel category discovery instead of UNOv2 | CIFAR100-50 task-aware clustering · CIFAR100-50 | clustering accuracy: 61.0 -> 62.2 | Helped |
Surprising findings
- Replacing mutual information with L2-distance between branch outputs drops CIFAR100-20 classification accuracy from 57.8% to 30.2%.
Limitations
- Classification of unlabeled classes is weak on harder splits; for example, ImageNet classification reaches only 26.5% and CIFAR100-50 reaches 21.6%.
- Relies on pretrained GloVe word vectors as the only source of semantic knowledge.
- Evaluation is limited to three small or medium benchmarks with a ResNet18 encoder.
Experimental setup
Datasets: CIFAR10 CIFAR100 ImageNet
Benchmarks: CIFAR10 CIFAR100-20 CIFAR100-50 ImageNet
Metrics: classification accuracy clustering accuracy
Baselines: k-means KCL MCL DTC RS RS+ UNO UNOv2 DEM f-CLSWGAN FREE CE-GZSL
Builds on: UNO Novel category discovery (RS/DTC) Zhang et al. 2022 clustering module GloVe
Closest papers by meaning
- SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery (NeurIPS 2025 · similarity 0.88)
- Prior-Constrained Association Learning for Fine-Grained Generalized Category Discovery (AAAI 2025 · similarity 0.87)
- HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts (ICLR 2025 · similarity 0.87)
- Deep Taxonomic Networks for Unsupervised Hierarchical Prototype Discovery (NeurIPS 2025 · similarity 0.87)
- Training-free Open-Vocabulary Semantic Segmentation via Diverse Prototype Construction and Sub-region Matching (AAAI 2025 · similarity 0.87)
- Envisioning Class Entity Reasoning by Large Language Models for Few-shot Learning (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.