Research map / Label-efficient and robust learning / Recognition and long-tail
Learning Along the Arrow of Time: Hyperbolic Geometry for Backward-Compatible Representation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HBCT, a backward-compatible representation learning method that lifts embeddings to the Lorentz hyperboloid, constrains new embeddings to lie in old embeddings' entailment cones, and uses an uncertainty-weighted RINCE contrastive alignment loss.
Problem
Backward-compatible representation learning: new embedding models must retrieve against legacy gallery embeddings without backfilling, but Euclidean/hyperspherical alignment losses force the new model to replicate poor old embeddings, hindering learning.
Approach
Lift embeddings into Lorentz hyperbolic space, use the hyperbolic norm-derived uncertainty of old embeddings to set entailment cone apertures and to adaptively weight a RINCE contrastive alignment loss, so the new model aligns more with confident old embeddings.
Key results
- HBCT improves CMC@1 compatibility (P_com) by 21.4% on average over the strongest Euclidean baseline, per the paper.
- HBCT yields a 44.8% increase in mAP compatibility over the strongest Euclidean baseline.
- In the extended-class CIFAR100 setting with ResNet18, HBCT reaches P_com of 0.495 (CMC@1) and 0.209 (mAP) versus 0.360 and 0.110 for Hot-Refresh.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HBCT instead of the strongest Euclidean baseline | Backward-compatible retrieval models · CIFAR100 and TinyImageNet, across four update scenarios | CMC@1 compatibility P_com: +21.4% compared to the strongest Euclidean baseline | Helped |
| Use HBCT instead of the strongest Euclidean baseline | Backward-compatible retrieval models · CIFAR100 and TinyImageNet, across four update scenarios | mAP compatibility P_com: +44.8% | Helped |
| Add the entailment cone loss (ablation) | HBCT · CIFAR100, Extended class | CMC@1 P_com: 0.435 -> 0.495 | Helped |
| Add the entailment cone loss (ablation) | HBCT · CIFAR100, Extended class | mAP P_com: 0.176 -> 0.209 | Helped |
| Replace InfoNCE with RINCE (ablation) | HBCT with entailment · CIFAR100, Extended class | CMC@1 P_com: 0.475 -> 0.495 | Helped |
| Replace InfoNCE with RINCE (ablation) | HBCT with entailment · CIFAR100, Extended class | mAP P_up: -0.009 -> -0.001 | Helped |
| Make curvature learnable rather than fixed at 1.0 (ablation) | HBCT · CIFAR100, Extended class | mAP P_comp: 0.207 -> 0.124 | Hurt |
| Set curvature K to 0.7 rather than 1.0 (ablation) | HBCT · CIFAR100, Extended class | mAP P_comp: 0.207 -> 0.211 | No clear effect |
| Set the new-model clipping threshold ζ_n to 1.2 rather than 1.0 (ablation) | HBCT · CIFAR100, Extended class | CMC@1 cross retrieval: 0.5525 -> 0.5705 | Helped |
| Replace RINCE with mean-distortion loss (ablation) | HBCT · CIFAR100, Extended class | mAP cross retrieval: 0.3982 -> 0.3970 | Hurt |
| Use Hot-Refresh instead of HBCT | ViT-B-16 update with ResNet18 old model · TinyImageNet, Both | CMC@1 P_com: 0.365 -> 0.348 | Hurt |
Surprising findings
- Hyperbolic HBCT with vanilla InfoNCE (no RINCE) already outperforms the best Euclidean baseline in the extended-class setting, per the ablation.
- Learnable curvature improves new-to-new retrieval but hurts cross (new-to-old) compatibility.
Limitations
- Evaluated only in supervised settings on small image benchmarks (CIFAR100, TinyImageNet); self-supervised and multimodal settings left for future work.
- Clipping threshold must grow with model updates, which may cause instability as norms grow (e.g., >=10).
- Removing the entailment loss causes a significant compatibility drop when the architecture changes to ViT-B-16.
Experimental setup
Datasets: CIFAR100 TinyImageNet ImageNet21k
Benchmarks: CIFAR100 TinyImageNet
Metrics: CMC@1 mAP P_com P_up
Baselines: l2 distance regularization BCT Hot-Refresh AdvBCT HOC
Builds on: BCT (backward-compatible training) RINCE entailment cone loss Hot-Refresh hyperbolic MLR
Closest papers by meaning
- Elastic Representations via Hyperbolic Geometry (NeurIPS 2026 · similarity 0.91)
- Hyperbolic Representation Learning: Revisiting and Advancing (ICML 2023 · similarity 0.91)
- Hyperbolic Embeddings of Supervised Models (NeurIPS 2024 · similarity 0.87)
- Temporal Geometry of Deep Networks: Hyperbolic Representations of Training Dynamics for Intrinsic Explainability (ICLR 2026 · similarity 0.87)
- Robust large-margin learning in hyperbolic space (NeurIPS 2020 · similarity 0.87)
- Weighted Embeddings for Low-Dimensional Graph Representation (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.