Research map / Label-efficient and robust learning / Recognition and long-tail
Deep Taxonomic Networks for Unsupervised Hierarchical Prototype Discovery
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Deep taxonomic networks fit a complete binary-tree mixture-of-Gaussians prior inside a VAE, derive that ELBO maximization approximates categorical-utility prototypicality, and evaluate unsupervised hierarchical prototype discovery across all tree levels.
Problem
Deep hierarchical clustering methods tie leaf nodes to the true number of classes and ignore informative intermediate prototypes, and they need retraining to change label granularity.
Approach
A VAE whose prior is a large complete-binary-tree mixture of Gaussians, with parent Gaussians derived by moment matching from children, so every node can act as a prototype; the ELBO is shown to approximate categorical utility, and a post-hoc annotation evaluation lets the frozen hierarchy be scored on any label set.
Key results
- On MNIST, DeepTaxonNet reaches 94.8% ACC vs 90.2% for TreeVAE; on Fashion it reaches 81.2% ACC vs 75.2% for Cobweb.
- On CIFAR-10 with contrastive learning, DeepTaxonNet achieves 67.97% ACC vs 52.98% for TreeVAE.
- Using the frozen CIFAR-20 hierarchy for 100 fine classes, DeepTaxonNet gets 26.36% ACC vs 11.98% for re-trained TreeVAE on CIFAR-100.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add both embedding-level and cluster-level contrastive losses (ablation) | DeepTaxonNet · CIFAR-10 | ACC: 38.59 -> 67.13 | Helped |
| Add both embedding-level and cluster-level contrastive losses (ablation) | DeepTaxonNet · CIFAR-10 | DP: 19.60 -> 42.74 | Helped |
| Use clustering-level contrastive loss only instead of no contrastive learning (ablation) | DeepTaxonNet · CIFAR-10 | ACC: 38.59 -> 65.41 | Helped |
| Use embedding-level contrastive loss only instead of no contrastive learning (ablation) | DeepTaxonNet · CIFAR-10 | ACC: 38.59 -> 40.92 | Helped |
| Add both entropy and symmetric-KL regularizers (ablation) | DeepTaxonNet · CIFAR-10 | ACC: 65.39 -> 67.13 | Helped |
| Add both entropy and symmetric-KL regularizers (ablation) | DeepTaxonNet · CIFAR-10 | DP: 40.38 -> 42.74 | Helped |
| Add entropy regularizer only (ablation) | DeepTaxonNet · CIFAR-10 | LP: 50.26 -> 52.85 | Helped |
| Add symmetric-KL regularizer only (ablation) | DeepTaxonNet · CIFAR-10 | LP: 50.26 -> 50.92 | Helped |
| Remove the symmetric-KL regularizer while retaining the entropy regularizer (ablation) | DeepTaxonNet · CIFAR-10 | ACC: 66.98 -> 67.13 | No clear effect |
| Apply contrastive learning instead of no contrastive learning (ablation) | DeepTaxonNet · CIFAR-10 | NMI: 20.07 -> 51.34 | Helped |
Surprising findings
- On MNIST, DeepTaxonNet has lower DP (76.6) than TreeVAE (87.9), attributed to bottom clusters capturing handwriting idiosyncrasies.
- Cobweb on raw pixels performs competitively on MNIST and Fashion, and gains little from VAE embeddings.
Limitations
- Pre-allocated complete binary tree prior assumes balanced feature splits, which may degrade on unbalanced data.
- Analysis uses diagonal isotropic Gaussian covariance for simplicity.
- Generative capabilities at multiple granularities are not explored.
Experimental setup
Datasets: MNIST Fashion-MNIST CIFAR-10 CIFAR-100 Omniglot
Benchmarks: MNIST Fashion-MNIST CIFAR-10 CIFAR-20 CIFAR-100
Metrics: ACC NMI dendrogram purity leaf purity probabilistic DP probabilistic LP
Baselines: Cobweb Cobweb + VAE DeepECT TreeVAE L2H-TEMI L2H-Turtle
Builds on: VAE VaDE TreeVAE Cobweb NT-Xent contrastive loss
Closest papers by meaning
- From Logits to Hierarchies: Hierarchical Clustering made Simple (ICML 2025 · similarity 0.87)
- Semantic-Guided Novel Category Discovery (AAAI 2024 · similarity 0.87)
- Classification Fields: Arbitrarily Fine Recursive Hierarchical Clustering From Few Examples (NeurIPS 2026 · similarity 0.86)
- SEAL: Semantic-Aware Hierarchical Learning for Generalized Category Discovery (NeurIPS 2025 · similarity 0.85)
- What Do You See in Common? Learning Hierarchical Prototypes over Tree-of-Life to Discover Evolutionary Traits (ICLR 2025 · similarity 0.85)
- HiLo: A Learning Framework for Generalized Category Discovery Robust to Domain Shifts (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.