Research map / Label-efficient and robust learning / Recognition and long-tail
Robust Hyperbolic Learning with Curvature-Aware Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes curvature-aware Riemannian optimization (tangent-based mapping of hyperbolic parameters when curvature changes), a Riemannian AdamW for the Lorentz model, and a tanh-based max-distance rescaling, applied to hierarchical metric learning, EEG classification, and hyperbolic CNN/VAE experiments.
Problem
Hyperbolic deep learning in the Lorentz model is prone to overfitting, instability (especially when curvature is learned and in low precision), and high computational cost.
Approach
Stagger updates so hyperbolic parameters are mapped to the new curvature via the tangent space at the origin (log map, then exp map with new curvature) and their gradients/momentum are transported accordingly; add a Riemannian AdamW implemented as a Lorentz centroid with the origin, and a smooth tanh-based max-norm rescaling in place of clipping.
Key results
- LHIER reports R@1 of 73.4 on CUB and 90.0 on Cars, with gains of 1.49% to 4.71% over HIER across the reported datasets.
- HyperMAtt reaches 68.12% on SSVEP and 77.98 AUC on ERN, state-of-the-art on those two datasets, with reported ~42x average per-epoch speed-up over MAtt.
- HECNN+ on ResNet-50 reaches 80.86% on CIFAR-100 versus 79.83% for HECNN, with reported ~48% VRAM reduction and ~66% runtime reduction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the HIER setup with Lorentz LHIER using learnable curvature, tangent-based mapping, and distance rescaling | Hierarchical metric learning · CUB, Cars, and SOP | Recall@k: CUB R@1 70.1 -> 73.4; Cars R@1 88.2 -> 90.0; SOP R@1 80.2 -> 81.9 | Helped |
| use HyperMAtt instead of MAtt | EEG classification · MI | accuracy: 74.71$\pm$5.01 -> 74.13$\pm$3.09 | Hurt |
| use HyperMAtt instead of MAtt | EEG classification · SSVEP | accuracy: 65.50$\pm$8.20 -> 68.12$\pm$2.63 | Helped |
| use HyperMAtt instead of MAtt | EEG classification · ERN | AUC: 76.01$\pm$2.28 -> 77.98$\pm$1.60 | Helped |
| use the proposed efficient hyperbolic architecture HECNN+ instead of HECNN | ResNet-50 image classification · CIFAR-100 | accuracy: 79.83 -> 80.86 | Helped |
| use the proposed efficient hyperbolic architecture HECNN+ instead of HECNN | ResNet-50 image classification · Tiny-ImageNet | accuracy: 66.30 -> 67.18 | Helped |
| add learnable curvature rather than fixing curvature (ablation) | HCNN+ on CIFAR-100 · CIFAR-100 | accuracy: 79.6 -> 80.86 | Helped |
| add tanh distance rescaling (ablation) | HCNN+ on CIFAR-100 · CIFAR-100 | accuracy: 80.13 -> 80.86 | Helped |
| add the curvature-aware optimization scheme (ablation) | HCNN+ with learned curvature on CIFAR-100 · CIFAR-100 | accuracy: $NaaN$ -> $80.86$ | Helped |
| replace RAdamW with RSGD (ablation) | LHIER · CUB | R@1: 64.2 -> 73.4 | Hurt |
| replace RAdamW with RAdam (ablation) | LHIER · CUB | R@1: 70.1 -> 73.4 | Hurt |
| fix curvature instead of learning it (ablation) | LHIER · CUB | R@1: 71.2 -> 73.4 | Hurt |
Surprising findings
- Learning curvature without the proposed optimization scheme causes the HCNN+ model to break down (NaN), per the ablation.
- On the MI EEG dataset, HyperMAtt (74.13%) is slightly below MAtt (74.71%), a reported drop of 0.78%.
Limitations
- Hyperbolic operations remain computationally expensive; approximations of exp/log maps are not studied.
- Theoretical reasoning is limited, as acknowledged by the authors.
- Alternatives for moving between manifolds are not explored and may depend on the task and loss.
Experimental setup
Datasets: CUB-200-2011 Cars-196 Stanford Online Products In-shop Clothes MI EEG SSVEP ERN CIFAR-100 Tiny-ImageNet CelebA Cora PubMed
Benchmarks: CUB-200-2011 Cars-196 Stanford Online Products MI SSVEP ERN CIFAR-100 Tiny-ImageNet CelebA Disease Airport PubMed
Metrics: Recall@k accuracy AUC FID VRAM time per epoch
Baselines: HIER ProxyNCA++ NSoftmax Hyp MAtt ShallowConvNet EEGNet HECNN HCNN HyboNet
Builds on: AdamW GeoOpt HIER HCNN MAtt
Closest papers by meaning
- Hyperbolic Aware Minimization: Implicit Bias for Sparsity (ICLR 2026 · similarity 0.89)
- Hyperbolic Vision Transformers: Combining Improvements in Metric Learning (CVPR 2022 · similarity 0.89)
- Adaptive Hyperbolic Kernels: Modulated Embedding in de Branges-Rovnyak Spaces (AAAI 2026 · similarity 0.89)
- Fast Hyperboloid Decision Tree Algorithms (ICLR 2024 · similarity 0.86)
- Fully Hyperbolic Convolutional Neural Networks for Computer Vision (ICLR 2024 · similarity 0.86)
- Task-Induced Riemannian Metrics for Vision Transformer Feature Spaces (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.