Research map / Label-efficient and robust learning / Recognition and long-tail

Hyperbolic Embeddings of Supervised Models

NeurIPS 2024 · Label-efficient and robust learning › Recognition and long-tail · approach: Contrastive representation theory · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes embedding supervised models (decision trees and boosted ensembles) in the Poincaré disk via a log-loss/hyperbolic distance link, monotonic decision trees, and a tempered-calculus "t-self" of the disk to improve numerical encoding near the boundary.

Problem

Hyperbolic embeddings have been used for data or unsupervised models, but there is no principled way to embed supervised models (e.g., decision trees and ensembles) in hyperbolic space while keeping the tree structure and prediction confidence readable and numerically accurate.

Approach

Map posterior confidence |ψ_LOG(p)| to radius in the Poincaré disk (log-loss canonical link equals hyperbolic distance to origin); embed monotonic subtrees of decision trees (MDTs) so each arc points outward; and generalize Riemann integration/Poincaré distance via tempered (t-) calculus to create a "t-self" of the disk that pushes points away from the boundary while preserving hyperbolicity.

Key results

• On the subset of UCI domains, MDT test error is statistically indistinguishable from the original DT on most domains (e.g., winered 18.39 vs 18.32, german 24.00 vs 23.90, qsar 12.98 vs 12.89), with differences on ionosphere, analcatdata_supreme and online_shoppers_intention.
• Visualization embedding error rho is small in general, e.g., ionosphere 15.21% for MDT #1 falling to 3.45% for MDT #5, and decreases across the ensemble on winered, german and qsar.
• The t-self with t=0.7 maps a point at r=1-10^-4 (k=4) to radius r(t)≈0.96, allowing a less risky encoding with k=2.

Measured effects reported in the paper

Surprising findings

• MDT test error is sometimes better than the DT (analcatdata_supreme) while on ionosphere and online_shoppers_intention the DT is better.
• Boosting reweighting moves the ensemble's MDT roots toward the disk center, so later MDTs embed with lower error.

Limitations

• Embedding algorithm is a heuristic modification of Sarkar's embedding with no optimization step, and MDTs can differ from the original DT in visualized structure.
• Decision trees are suited mainly to tabular data and should be used with caution.
• Embedding errors are non-zero, and the tree sizes are capped at 200 nodes in experiments.

Experimental setup

Datasets: UCI ionosphere winered german analcatdata_supreme abalone qsar online_shoppers_intention hardware

Benchmarks: UCI ionosphere winered german analcatdata_supreme abalone qsar online_shoppers_intention hardware

Metrics: test error embedding error rho (%) paired t-test p-value

Baselines: decision tree (DT) Sarkar's embedding

Builds on: Sarkar's tree embedding C4.5 AdaBoost t-algebra (tempered arithmetic)

Closest papers by meaning

• Random Laplacian Features for Learning with Hyperbolic Space (ICLR 2023 · similarity 0.91)
• Bridging ML and algorithms: comparison of hyperbolic embeddings (ICLR 2026 · similarity 0.91)
• Representation Tradeoffs for Hyperbolic Embeddings (ICML 2018 · similarity 0.91)
• Weighted Embeddings for Low-Dimensional Graph Representation (AAAI 2025 · similarity 0.90)
• Low-distortion and GPU-compatible Tree Embeddings in Hyperbolic Space (ICML 2025 · similarity 0.89)
• Learning Structured Representations with Hyperbolic Embeddings (NeurIPS 2024 · similarity 0.88)