Research map / Label-efficient and robust learning / Recognition and long-tail
Learning Structured Representations with Hyperbolic Embeddings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HypStructure, a hyperbolic (Poincaré ball) structured regularizer that combines a hyperbolic CPCC loss over the full label tree with a centering loss, improving hierarchy embedding, classification and OOD detection.
Problem
Existing structured representation learning that embeds label hierarchies via Euclidean CPCC regularization cannot embed tree metrics without distortion, especially in low dimensions, and treats internal nodes poorly.
Approach
Replace the Euclidean l2-CPCC regularizer with a hyperbolic CPCC computed with Poincaré-ball distances over prototypes of all tree nodes (including internal nodes), plus a hyperbolic centering loss that places the root near the origin; this is combined with CE or SupCon and linked to an eigenspectrum analysis explaining OOD detection gains.
Key results
- HypStructure reduces hierarchy distortion over Flat by up to 59.4% and over l2-CPCC by up to 45.4%.
- HypStructure improves kNN fine/coarse accuracy by up to 2.2% over Flat and up to 0.8% over l2-CPCC.
- On ImageNet100, HypStructure achieves 93.83 average OOD AUROC versus 92.74 for KNN+ and 91.33 for l2-CPCC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HypStructure instead of SSD+ | OOD detection with CIFAR10 as ID · CIFAR10 | AUROC: 97.38 -> 97.75 | Helped |
| Use HypStructure instead of KNN+ | OOD detection with ImageNet100 as ID · ImageNet100 | AUROC: 92.74 -> 93.83 | Helped |
| Use HypStructure rather than Flat representations | hierarchical representation learning · CIFAR10, CIFAR100, and ImageNet100 | distortion: upto 59.4% | Helped |
| Use HypStructure rather than ℓ2-CPCC | hierarchical representation learning · CIFAR10, CIFAR100, and ImageNet100 | distortion: upto 45.4% | Helped |
| Use HypStructure rather than Flat | kNN classification on fine and coarse classes · CIFAR10, CIFAR100, and ImageNet100 | accuracy: upto 2.2% | Helped |
| Use HypStructure rather than ℓ2-CPCC | kNN classification on fine and coarse classes · CIFAR10, CIFAR100, and ImageNet100 | accuracy: upto 0.8% | Helped |
| Use HypStructure rather than Flat (SupCon) and ℓ2-CPCC | OOD detection · CIFAR10, CIFAR100, and ImageNet100 OOD evaluations | AUROC: upto 2% | Helped |
Surprising findings
- Leaf-only l2-CPCC optimization yields even larger tree-metric distortion than full-tree l2-CPCC on CIFAR10, and l2-CPCC achieves only 76.67 average AUROC on CIFAR10 OOD detection versus 97.75 for HypStructure.
Limitations
- Requires an external or constructed label hierarchy; noisy hierarchies may hurt performance.
- Only the Poincaré ball model is used; Lorentz and other models are left to future work.
- The eigenspectrum analysis relies on idealized assumptions (perfect CPCC maximization, standardized features, balanced trees) approximated by the theory.
Experimental setup
Datasets: CIFAR10 CIFAR100 ImageNet100 SVHN Places365 Textures LSUN iSUN SUN iNaturalist
Benchmarks: CIFAR10 CIFAR100 ImageNet100
Metrics: Gromov hyperbolicity delta_rel CPCC kNN accuracy AUROC Borda count
Baselines: Flat (SupCon) l2-CPCC SSD+ KNN+ CIDER
Builds on: l2-CPCC structured representation learning Supervised Contrastive loss Mahalanobis OOD score Poincaré embeddings
Closest papers by meaning
- Learning Structured Representations by Embedding Class Hierarchy (ICLR 2023 · similarity 0.90)
- Hyperbolic Embeddings of Supervised Models (NeurIPS 2024 · similarity 0.88)
- Hyperbolic Category Discovery (CVPR 2025 · similarity 0.87)
- Learning Structured Representations by Embedding Class Hierarchy with Fast Optimal Transport (ICLR 2025 · similarity 0.86)
- Fully Hyperbolic Convolutional Neural Networks for Computer Vision (ICLR 2024 · similarity 0.86)
- Compositional Entailment Learning for Hyperbolic Vision-Language Models (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.