Research map / Graph learning / Graph self-supervised learning
Graph Geometry-Preserving Autoencoders
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GGAE, an autoencoder regularizer that uses a graph-Laplacian-based Riemannian distortion measure (with mini-batch Batch-Kernel approximation) to preserve the geometry implied by a similarity graph in the latent space.
Problem
Autoencoders often distort the latent geometry of data manifolds, and existing geometry-preserving methods assume a Euclidean ambient metric that may not reflect semantic structure; graph-based alternatives are either global-distance-dependent or fragile to missing edges.
Approach
Derive the ambient-space metric implicitly from a similarity graph via graph Laplacian estimates of J G^-1 J^T, and use the resulting coordinate-free Riemannian distortion as an autoencoder regularizer, with a mini-batch Batch-Kernel approximation for scalability. It relies on local metrics rather than global geodesic distances.
Key results
- GGAE achieves the best average rank on kNN, Spearman, KL0.01, KL0.1, KL1 and dynamics error across datasets (e.g., average rank 1.2 on Spear/KL metrics) in Table 1.
- On Swiss Roll, Rotating MNIST and robot images, only GGAE (and SPAE-graph in some cases) recovers the ideal graph geometry, while other baselines produce distorted or disconnected latents.
- With sparse graphs (p=0.85, 0.9 edge removal), GGAE keeps graph geometry preservation while SPAE-graph degrades drastically.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GGAE instead of SPAE-graph | Graph geometry-preserving autoencoders · All datasets | Spear rank: 1.8 ± 1.0 -> 1.2 ± 0.5 | Helped |
| Use GGAE instead of SPAE-graph | Graph geometry-preserving autoencoders · All datasets | KL0.01 rank: 3.2 ± 3.2 -> 1.2 ± 0.5 | Helped |
| Use GGAE instead of SPAE-graph | Graph geometry-preserving autoencoders · All datasets | KL0.1 rank: 3.2 ± 2.6 -> 1.2 ± 0.5 | Helped |
| Use GGAE instead of SPAE-graph | Graph geometry-preserving autoencoders · All datasets | KL1 rank: 2.8 ± 2.2 -> 1.2 ± 0.5 | Helped |
| Use GGAE instead of SPAE-graph | Graph geometry-preserving autoencoders · All datasets | dynamics rank: 3 -> 1 | Helped |
| Use Batch-Kernel instead of Laplacian-Slicing (ablation) | GGAE · Graph geometry preservation | distortion estimation | Helped |
| Use local metric-based regularization rather than global distance-based regularization | GGAE versus SPAE-graph · Swiss Roll and robot image datasets with sparse graphs | graph geometry preservation | Helped |
Surprising findings
- Laplacian-Slicing mini-batch estimator fails to preserve geometry due to high variance, even though it is unbiased, while the biased Batch-Kernel works better.
Limitations
- Bandwidth parameter h lacks a canonical tuning strategy and must be tuned per dataset
- Requires a similarity graph, which may be unavailable for some datasets
- Reconstruction sometimes sacrificed to obtain loop-shaped or graph-preserving latents (e.g., Rotating MNIST)
Experimental setup
Datasets: Swiss Roll dSprites Rotating MNIST robot manipulator images
Benchmarks: Swiss Roll dSprites Rotating MNIST robot manipulator latent dynamics
Metrics: reconstruction error kNN Spearman correlation KL divergence (KL0.01, KL0.1, KL1) dynamics prediction MSE
Baselines: AE VAE IRAE GeomAE SPAE SPAE-graph TopoAE TopoAE-graph NRAE-graph GRAE
Builds on: Perraul-Joncas & Meila (2013) Laplacian-based metric estimation IRAE Hein et al. (2007) graph Laplacian convergence SPAE Hafner et al. Dreamer latent dynamics
Closest papers by meaning
- Geometrically regularized autoencoders for non-Euclidean data (ICLR 2023 · similarity 0.91)
- Regularized Autoencoders for Isometric Representation Learning (ICLR 2022 · similarity 0.91)
- Learning Flat Latent Manifolds with VAEs (ICML 2020 · similarity 0.90)
- A Graph Regularized Deep Neural Network for Unsupervised Image Representation Learning (CVPR 2017 · similarity 0.90)
- Euclidean Embedding of Data Using Local Distances (NeurIPS 2026 · similarity 0.87)
- The Geometric Wall: Manifold Structure Predicts Layerwise Sparse Autoencoder Scaling Laws (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.