Research map / Label-efficient and robust learning / Recognition and long-tail
Adaptive Part Learning for Fine-Grained Generalized Category Discovery: A Plug-and-Play Enhancement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes APL, a plug-and-play module that discovers consistent object parts via shared learnable part queries guided by DINO part priors, trained with an all-min contrastive loss and diversity loss, replacing the CLS token in GCD frameworks.
Problem
Generalized category discovery on fine-grained images suffers a trade-off between discriminability and generalization when relying on the global DINO CLS token representation.
Approach
Discover consistent, one-to-one corresponding object parts across images using shared learnable part queries initialized/guided by DINO attention priors, and learn them with an all-min contrastive loss that constrains all parts of positives but only the least similar part of negatives, so discriminative parts separate classes while shared parts transfer knowledge.
Key results
- Ours (SimGCD) improves All accuracy on Stanford Cars from 53.8 to 60.1 and on average adds 3.7% over SimGCD across fine-grained datasets.
- Ours (SPTNet) reaches 47.3 All / 37.0 Novel on Herbarium-19 vs. SPTNet's 43.4 / 35.2, with +7.9 Known gain.
- On Herbarium-19, Ours (CMS†) improves All accuracy by 2.5 points over the reproduced CMS result.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add APL to SimGCD | SimGCD · CUB | accuracy: 60.3 -> 64.5 | Helped |
| add APL to SimGCD | SimGCD · Stanford Cars | accuracy: 53.8 -> 60.1 | Helped |
| add APL to SPTNet | SPTNet · FGVC-Aircraft | accuracy: 59.3 -> 60.9 | Helped |
| add APL to CMS† | CMS† · Herbarium-19 | accuracy: 35.9 -> 38.4 | Helped |
| add APL to SimGCD | SimGCD · CIFAR-100 | accuracy: 82.3 -> 80.9 | Hurt |
| add per-image k-means patch clustering for part discovery (ablation) | SimGCD · CUB | accuracy: 53.8 -> 51.0 | Hurt |
| add DINO part priors as parts (ablation) | SimGCD · CUB | accuracy: 53.8 -> 55.1 | Helped |
| add learnable queries to DINO part priors (ablation) | SimGCD with DINO part priors · CUB | accuracy: 55.1 -> 58.5 | Helped |
| replace soft assignment with Gumbel-Softmax hard assignment (ablation) | Learnable-query part discovery · CUB | accuracy: 58.5 -> 60.1 | Helped |
| add standard part contrastive loss (ablation) | SimGCD + part discovery · CUB | accuracy: 55.1 -> 55.2 | No clear effect |
| add min-max contrastive loss (ablation) | SimGCD + part discovery · CUB | accuracy: 55.1 -> 57.8 | Helped |
| add diversity loss (ablation) | SimGCD + part discovery · CUB | accuracy: 55.1 -> 56.7 | Helped |
Surprising findings
- Naive k-means clustering of patches per image lowers SimGCD performance (53.8 to 51.0 All), indicating local information can add noise without consistency.
Limitations
- Gains on generic datasets (CIFAR-10/100, ImageNet-100) are smaller, since they need fewer fine-grained parts.
- Part queries are a fixed number T and part discovery adds an extra Gumbel-softmax assignment step.
Experimental setup
Datasets: CUB Stanford Cars FGVC-Aircraft Herbarium-19 CIFAR-10 CIFAR-100 ImageNet-100
Benchmarks: CUB Stanford Cars FGVC-Aircraft Herbarium-19 CIFAR-10 CIFAR-100 ImageNet-100
Metrics: clustering accuracy (All, Known, Novel)
Baselines: SimGCD SPTNet CMS GCD DCCL PromptCAL PMI InfoSieve SelEx
Builds on: DINO SimGCD SPTNet CMS GCD
Closest papers by meaning
- SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery (NeurIPS 2025 · similarity 0.87)
- Solving the Catastrophic Forgetting Problem in Generalized Category Discovery (CVPR 2024 · similarity 0.87)
- SPTNet: An Efficient Alternative Framework for Generalized Category Discovery with Spatial Prompt Tuning (ICLR 2024 · similarity 0.87)
- HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts (ICLR 2025 · similarity 0.86)
- Multi-Part Object Representations via Graph Structures and Co-Part Discovery (NeurIPS 2026 · similarity 0.86)
- DebGCD: Debiased Learning with Distribution Guidance for Generalized Category Discovery (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.