Research map / Label-efficient and robust learning / Recognition and long-tail
Prototypical Hash Encoding for On-the-Fly Fine-Grained Category Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Prototypical Hash Encoding (PHE), which learns multiple prototypes per category and maps them to hash centers separated by a Gilbert-Varshamov-derived minimum distance, for fine-grained on-the-fly category discovery.
Problem
On-the-fly category discovery (OCD) must assign streaming test instances to known or unknown categories online; existing hash-based methods such as SMILE suffer from "high sensitivity" of hash codes, producing inaccurate category counts and poor accuracy, especially for fine-grained classes.
Approach
Perform representation learning in high-dimensional feature space with multi-prototype (ProtoPNet/ProtoPFormer-style) classification, map each category's mean prototype to a hash center, and enforce a minimum Hamming separation between centers (from the Gilbert-Varshamov bound) with Hamming-ball-based online inference.
Key results
- PHE achieves 36.4 All accuracy on CUB and 31.3 on Stanford Cars, versus 32.2 and 26.2 for SMILE.
- Averaged over eight datasets, PHE reaches 36.3 All / 59.1 Old / 25.1 New on the first four datasets and 37.2 All on the iNaturalist four, improving All accuracy by about 5.3% over SMILE as stated in the abstract.
- At 64-bit hash codes, SMILE's estimated class count on CUB rises to 2910 while PHE stays at 493, indicating reduced high sensitivity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add prototype representation learning (L_p) (ablation) | PHE · CUB-200 | All ACC: 34.9 -> 36.4 | Helped |
| add prototype representation learning (L_p) (ablation) | PHE · Stanford Cars | All ACC: 28.9 -> 31.3 | Helped |
| add hash-center optimization (L_c) (ablation) | PHE · CUB-200 | All ACC: 32.0 -> 36.4 | Helped |
| add hash-center optimization (L_c) (ablation) | PHE · Stanford Cars | All ACC: 24.1 -> 31.3 | Helped |
| add hash-feature encoding loss (L_f) (ablation) | PHE · CUB-200 | All ACC: 34.1 -> 36.4 | Helped |
| add hash-feature encoding loss (L_f) (ablation) | PHE · Stanford Cars | All ACC: 26.0 -> 31.3 | Helped |
| replace handcrafted fixed hash points with learned category-specific hash centers (ablation) | PHE · CUB-200 | All ACC: 35.4 -> 36.4 | Helped |
| replace handcrafted fixed hash points with learned category-specific hash centers (ablation) | PHE · Stanford Cars | All ACC: 30.0 -> 31.3 | Helped |
| use prototype learning instead of linear classification (ablation) | PHE · CUB-200 | All ACC: 35.6 -> 36.4 | Helped |
| use prototype learning instead of linear classification (ablation) | PHE · Stanford Cars | All ACC: 29.4 -> 31.3 | Helped |
| use prototype learning instead of supervised contrastive classification (ablation) | PHE · CUB-200 | All ACC: 35.3 -> 36.4 | Helped |
| use prototype learning instead of supervised contrastive classification (ablation) | PHE · Stanford Cars | All ACC: 29.5 -> 31.3 | Helped |
Surprising findings
- SMILE's On-the-fly accuracy drops sharply as hash length grows (e.g., CUB All from 32.2 at 12-bit to 13.2 at 256-bit), even though offline clustering accuracy rises.
Limitations
- Accuracy on new (unseen) categories remains low, with New accuracy often around 11-27% on the fine-grained datasets.
- Evaluated only on image classification datasets with a DINO ViT-B/16 backbone.
Experimental setup
Datasets: CUB-200 Stanford Cars Oxford-IIIT Pet Food-101 iNaturalist 2017 (Fungi, Arachnida, Animalia, Mollusca)
Benchmarks: CUB-200 Stanford Cars Oxford-IIIT Pet Food-101 iNaturalist Fungi iNaturalist Arachnida iNaturalist Animalia iNaturalist Mollusca
Metrics: clustering accuracy (All, Old, New) estimated number of classes
Baselines: SMILE SLC RankStat WTA DPN OrthoHash CSQ MDSH SimGCD
Builds on: SMILE ProtoPFormer ProtoPNet SimGCD
Closest papers by meaning
- Assignment-Driven Hash Learning in a Hyper-Semantic Space for On-the-Fly Category Discovery (CVPR 2026 · similarity 0.94)
- Learning Attribute-Aware Hash Codes for Fine-Grained Image Retrieval via Query Optimization (ICML 2025 · similarity 0.86)
- An Asymmetric Augmented Self-Supervised Learning Method for Unsupervised Fine-Grained Image Hashing (CVPR 2024 · similarity 0.85)
- Characteristics Matching Based Hash Codes Generation for Efficient Fine-grained Image Retrieval (CVPR 2024 · similarity 0.85)
- KALAHash: Knowledge-Anchored Low-Resource Adaptation for Deep Hashing (AAAI 2025 · similarity 0.84)
- Effective Comparative Prototype Hashing for Unsupervised Domain Adaptation (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.