Research map / Learning theory and algorithms / Matrix and tensor methods

Building Transformation Layers for Riemannian Neural Networks

NeurIPS 2026 · Learning theory and algorithms › Matrix and tensor methods · approach: Geometry of low-rank and SPD structure · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a geometry-agnostic framework for building fully connected and convolutional layers on Riemannian manifolds using point-to-hyperplane pseudo-distances, instantiated on ten manifolds (hyperbolic, SPD, Grassmannian).

Problem

Existing FC and convolutional layers for manifold-valued representations are tied to specific manifolds and rely on special structures (Poincaré, gyro, symmetric-space), so no general, flexible construction exists across geometries.

Approach

Generalize Euclidean FC layers via Riemannian point-to-hyperplane pseudo-distances over multiple tangent spaces, needing only exp/log maps; the resulting layer is Y = exp_E(sum_i <log_{P_i}(X), A_i> B_i), with parameters trivialized as one tangent vector and one scalar per output dimension, so Euclidean optimizers suffice. Several prior FC layers are special cases.

Key results

• HFC-H reaches 92.20% AUC on Cora and 95.18% on Airport, the best among compared hyperbolic FC layers in the HNN setting.
• SPDNN-LEM achieves 81.16% on HDM05 and 86.72% on NTU60, outperforming SPDNet by up to 16.59% and 20.36% respectively.
• On Cora, NestFC uses 89.3x as many parameters and 3.8x as much peak memory as HFC-H.

Surprising findings

• Tangent-space transformations remain competitive on higher-delta Pubmed, suggesting tangent approximations depend on data hyperbolicity.

Limitations

• Requires closed-form or tractable exp/log maps; not applicable to manifolds lacking them.
• Uses a pseudo-distance rather than the true point-to-hyperplane distance.
• Exp/log maps on incomplete manifolds (PEM, BWM) and Grassmannian cut locus require numerical regularization.

Experimental setup

Datasets: Disease Airport Pubmed Cora Radar HDM05 FPHA NTU60

Benchmarks: Disease Airport Pubmed Cora Radar HDM05 FPHA NTU60

Metrics: AUC accuracy number of parameters peak GPU memory training time per epoch

Baselines: Möbius Einstein LFC NestFC Poincaré FC SPDNet SPDNetBN RResNet LieBN MLR GyroSPD++ GrNet

Builds on: Poincaré FC layer gyro SPD FC layers SPDNet RResNet NestFC GyroSPD++

Closest papers by meaning

• Riemannian Residual Neural Networks (NeurIPS 2023 · similarity 0.90)
• Matrix Manifold Neural Networks++ (ICLR 2024 · similarity 0.89)
• RMLR: Extending Multinomial Logistic Regression into General Geometries (NeurIPS 2024 · similarity 0.88)
• Neural networks on Symmetric Spaces of Noncompact Type (ICLR 2025 · similarity 0.87)
• Consistent Geometric Deep Learning via Hilbert Bundles and Cellular Sheaves (NeurIPS 2026 · similarity 0.87)
• Residual Deep Gaussian Processes on Manifolds (ICLR 2025 · similarity 0.87)