Research map / Label-efficient and robust learning / Recognition and long-tail
Contrastive Mean-Shift Learning for Generalized Category Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Contrastive Mean-Shift (CMS) learning, which trains a DINO ViT-B/16 image encoder with a single-step kNN-based mean-shift contrastive objective and clusters with iterative mean shift plus agglomerative clustering with jointly estimated class number, reaching SOTA on six GCD benchmarks.
Problem
Generalized category discovery: partition a partially labeled image collection into known and unknown classes when the total number of classes is unknown, which prior methods typically handle by using the ground-truth K.
Approach
Integrates mean-shift (kNN-approximated, GPU-friendly variant) into contrastive learning: each embedding is replaced by its single-step kNN mean-shifted version before the contrastive loss, and at inference mean shift is iterated on the embedding space with the number of clusters estimated via agglomerative clustering.
Key results
- Achieves SOTA on five of six datasets with ground-truth K, with gains of 4.7 points on Stanford Cars and 3.1 points on CUB over prior methods.
- Without ground-truth K, reaches 79.6 All on CIFAR100 and 64.4 All on CUB, comparable to methods that use the GT K.
- Ablation on CUB shows iterative mean shift at inference gives additional gains without further training, saturating after a certain number of iterations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add contrastive mean-shift learning | DINO ViT-B/16 for generalized category discovery · CIFAR100, ImageNet100, CUB, Stanford Cars, FGVC Aircraft, Herbarium19 | All accuracy: 82.3 vs 81.2; 84.7 vs 83.1; 68.2 vs 63.5; 56.9 vs 53.8; 56.0 vs 54.2; 36.4 vs 44.0 | Mixed |
| add contrastive mean-shift learning | DINO ViT-B/16 for generalized category discovery without known K · CIFAR100, ImageNet100, CUB, Stanford Cars, FGVC Aircraft, Herbarium19 | All accuracy: 79.6 vs 75.6; 81.3 vs 83.0; 64.4 vs 62.0; 51.7 vs 42.4; 55.2 vs 43.3; 37.4 vs 42.0 | Mixed |
| add CMS learning rather than using embeddings without mean shift (ablation) | CMS · CIFAR100, ImageNet100, CUB, Stanford Cars | All accuracy: 71.5 -> 81.1; 74.1 -> 83.4; 51.2 -> 66.7; 37.9 -> 54.5 | Helped |
| add iterative mean shift at inference (ablation) | CMS · CIFAR100, ImageNet100, CUB, Stanford Cars | All accuracy: 71.5 -> 71.6; 74.1 -> 80.3; 51.2 -> 58.7; 37.9 -> 40.8 | Mixed |
| replace single-step mean shift with iterative mean shift (ablation) | CMS · CIFAR100, ImageNet100, CUB, Stanford Cars | All accuracy: 80.1 -> 82.3; 84.1 -> 84.7; 68.2 -> 68.2; 56.1 -> 56.9 | Mixed |
| replace the kNN neighborhood with a uniform kernel (ablation) | CMS · CIFAR100, ImageNet100, CUB, Stanford Cars, FGVC Aircraft, Herbarium19 | All accuracy: 75.2 vs 82.3; 76.9 vs 84.7; 53.5 vs 68.2; 34.3 vs 56.9; 39.9 vs 56.0; 36.0 vs 36.4 | Hurt |
| replace the kNN neighborhood with a Gaussian kernel (ablation) | CMS · CIFAR100, ImageNet100, CUB, Stanford Cars, FGVC Aircraft, Herbarium19 | All accuracy: 72.2 vs 82.3; 67.2 vs 84.7; 45.5 vs 68.2; 34.9 vs 56.9; 36.6 vs 56.0; 25.6 vs 36.4 | Hurt |
| replace kNN retrieval and mean aggregation with random retrieval and mean aggregation (ablation) | CMS · ImageNet100 and CUB | All accuracy: 82.6 vs 84.7; 58.1 vs 68.2 | Hurt |
| replace mean aggregation with learnable attentive aggregation (ablation) | CMS · ImageNet100 and CUB | All accuracy: 82.2 vs 84.7; 63.6 vs 68.2 | Hurt |
| add CMS rather than the Vaze et al. method (ablation) | unsupervised category discovery · CIFAR100, ImageNet100, CUB | accuracy: +5.7%p; +3.8%p; +4.5%p | Helped |
| use labeled images in addition to unsupervised training (ablation) | CMS · CIFAR100, ImageNet100, CUB | kNN Recall@8: 67.4 -> 87.9; 88.1 -> 90.2; 29.5 -> 82.6 | Helped |
Surprising findings
- Fixed-radius (uniform or Gaussian) mean-shift kernels significantly degrade performance because the blurred embedding space makes embeddings indistinguishable as training progresses.
- Random kNN aggregation deteriorates performance as training progresses, showing neighbor quality matters for the mean-shift objective.
Limitations
- Convergence of the kNN-based mean shift is not guaranteed theoretically; only empirically observed to roughly converge.
- Estimated cluster numbers show notable errors on some datasets (e.g., 16% error on ImageNet100 and CUB, 20.4% on Stanford Cars).
Experimental setup
Datasets: CIFAR100 ImageNet100 CUB-200-2011 Stanford Cars FGVC Aircraft Herbarium 19
Benchmarks: CIFAR100 ImageNet100 CUB-200-2011 Stanford Cars FGVC Aircraft Herbarium 19
Metrics: clustering accuracy (All/Old/Novel) estimated number of clusters error
Baselines: GCD PromptCAL SimGCD PIM GPC DCCL ORCA UNO+ RankStats+ Agglomerative clustering
Builds on: GCD (Vaze et al.) mean shift supervised contrastive learning SimGCD PromptCAL
Closest papers by meaning
- GeCC: Generalized Contrastive Clustering with Domain Shifts Modeling (AAAI 2025 · similarity 0.87)
- Generalized Category Discovery under Domain Shift: A Frequency Domain Perspective (NeurIPS 2025 · similarity 0.87)
- DebGCD: Debiased Learning with Distribution Guidance for Generalized Category Discovery (ICLR 2025 · similarity 0.86)
- SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery (NeurIPS 2025 · similarity 0.86)
- Towards Understanding Parametric Generalized Category Discovery on Graphs (ICML 2025 · similarity 0.86)
- HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.