Research map / 3D vision / Face and shape
Neural Isometries: Taming Transformations for Equivariant ML
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Neural Isometries (NIso), a self-supervised autoencoder that regularizes latent functional maps between observations to be isometries commuting with a learned operator, enabling simple equivariant heads and camera pose regression.
Problem
Symmetries in observation space (camera motion, homographies, conformal deformations) often lack analytical group structure, making equivariant networks hand-crafted and hard to build. The paper asks how to learn latent spaces where such transformations become tractable linear maps.
Approach
Learn an autoencoder whose latent functional maps between transformed observations are constrained to be isometries: orthogonal with respect to a learned mass matrix and commuting with a learned PSD operator, so the maps become block-diagonal in the operator eigenbasis; a multiplicity loss keeps the operator non-trivial.
Key results
- On homography-perturbed MNIST, NIso reaches 92.52% accuracy (97.38% with triplet regime), on par with handcrafted homConv (95.71%) and LieDecomp (98.30%); the NFT gets 41.93%.
- On conformal SHREC '11, NIso achieves 90.26% accuracy, above MobiusConv (86.5%) and NFT (83.24%).
- On CO3Dv2 pose estimation, NIso achieves lowest mean ATE at frame skips 1-9 (e.g., 0.081 at skip 9 vs 0.104 for NFT), though NFT and ViT are slightly better at skip 0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use triplet training instead of pairwise training (ablation) | NIso · Homography-Perturbed MNIST | accuracy: 92.52 ($\pm$ 0.91) -> 97.38 ($\pm$ 0.23) | Helped |
| Add the equivariance loss (ablation) | NIso · Homography-Perturbed MNIST | accuracy: 77.30 ($\pm$ 2.56) -> 92.52 ($\pm$ 0.91) | Helped |
| Add the multiplicity loss (ablation) | NIso · Homography-Perturbed MNIST | accuracy: 45.27 ($\pm$ 1.20) -> 92.52 ($\pm$ 0.91) | Helped |
| Use NIso rather than NFT | Homography-Perturbed MNIST classification · Homography-Perturbed MNIST | accuracy: 41.93 ($\pm$ 0.84) -> 92.52 ($\pm$ 0.91) | Helped |
| Use NIso rather than homConv | Homography-Perturbed MNIST classification · Homography-Perturbed MNIST | accuracy: 95.71 ($\pm$ 0.09) -> 92.52 ($\pm$ 0.91) | Hurt |
| Use NIso rather than LieDecomp | Homography-Perturbed MNIST classification · Homography-Perturbed MNIST | accuracy: 98.30 ($\pm$ 0.10) -> 92.52 ($\pm$ 0.91) | Hurt |
| Use NIso rather than NFT | Conformal shape classification · Conformal SHREC '11 | accuracy: 83.24 ($\pm$ 2.03) -> 90.26 ($\pm$ 1.27) | Helped |
| Use NIso rather than Möbius Convolutions | Conformal shape classification · Conformal SHREC '11 | accuracy: 86.5 -> 90.26 ($\pm$ 1.27) | Helped |
| Remove the learned operator Ω (ablation) | NIso · CO3D Sequence ATE, frame skip 0 | ATE: 0.023 ($\pm$ 0.001) -> 0.027 ($\pm$ 0.001) | Hurt |
| Remove the learned operator Ω (ablation) | NIso · CO3D Sequence ATE, frame skips 1, 3, 5, 7, and 9 | ATE: NIso: 0.031 ($\pm$ 0.001), 0.044 ($\pm$ 0.001), 0.057 ($\pm$ 0.002), 0.071 ($\pm$ 0.002), 0.081 ($\pm$ 0.002); w/o Ω: 0. | Hurt |
| Use NIso rather than NFT | Camera pose estimation · CO3D Sequence ATE, frame skips 1, 3, 5, 7, and 9 | ATE: NIso: 0.031 ($\pm$ 0.001), 0.044 ($\pm$ 0.001), 0.057 ($\pm$ 0.002), 0.071 ($\pm$ 0.002), 0.081 ($\pm$ 0.002); NFT: 0.03 | Helped |
| Use NIso rather than ViT | Camera pose estimation · CO3D Sequence ATE, frame skips 1, 3, 5, 7, and 9 | ATE: NIso: 0.031 ($\pm$ 0.001), 0.044 ($\pm$ 0.001), 0.057 ($\pm$ 0.002), 0.071 ($\pm$ 0.002), 0.081 ($\pm$ 0.002); ViT: 0.04 | Helped |
Surprising findings
- Removing the multiplicity loss drops homNIST accuracy sharply from 92.52% to 45.27%, showing the sparsity regularizer is critical.
- The NFT baseline performs poorly on homNIST despite being a related method, with latent maps that are not sparse or condensed.
Limitations
- Cannot learn and transfer operators across domains with varying connectivity, limiting geometry processing and graph tasks.
- Does not explicitly handle partiality or occlusion, which likely degrades pose estimation.
- Fails when foreground and background depths are very different, as depth maps become ill-defined.
Experimental setup
Datasets: MNIST homNIST SHREC '11 CO3Dv2 ImageNet
Benchmarks: homNIST SHREC '11 conformal CO3Dv2 camera pose
Metrics: classification accuracy absolute trajectory error (ATE) equivariance error
Baselines: NFT homConv LieDecomp MobiusConv DINOv2 BEiT ViT transformer baseline AE w/ aug.
Builds on: functional maps Neural Fourier Transform vector neurons DUSt3R
Closest papers by meaning
- Self-supervised learning of Split Invariant Equivariant representations (ICML 2023 · similarity 0.89)
- Frame Averaging for Equivariant Shape Space Learning (CVPR 2022 · similarity 0.89)
- Learning Symmetric Embeddings for Equivariant World Models (ICML 2022 · similarity 0.89)
- Connecting Neural Models Latent Geometries with Relative Geodesic Representations (NeurIPS 2025 · similarity 0.88)
- Grounding Continuous Representations in Geometry: Equivariant Neural Fields (ICLR 2025 · similarity 0.87)
- In-Context Symmetries: Self-Supervised Learning through Contextual World Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.