Research map / 3D vision / Face and shape
Learning with 3D rotations, a hitchhiker's guide to SO(3)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
placeholder
Problem
Choosing a 3D rotation representation for neural network regression is difficult because representation properties (continuity, double cover, dimensionality) affect gradient-based learning when rotations appear in the model's input or output.
Approach
Analyzes rotation representations through the lens of the continuity (Lipschitz behavior) of the map g: SO(3)->R, showing that low-dimensional (three- or four-parameter) representations inevitably introduce discontinuities when rotations are outputs, and that loss changes or half-space maps cannot fix this; recommends SVD and GSO (6D/9D) for rotation estimation and high-dimensional inputs for feat
Key results
- On point-cloud rotation estimation, SVD performed best, followed by GSO; low-dimensional representations performed worse.
- On YCB-Video 6D pose estimation, Euler angles performed notably worse than SVD and GSO, while quaternions performed similarly to the continuous representations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use R9+SVD instead of R6+GSO | rotation estimation from point clouds · Point cloud alignment | test performance | Helped |
| use a high-dimensional representation instead of a low-dimensional representation | rotation estimation · Cube rotation from images | Chordal distance | Helped |
| add a half-space map to quaternions (ablation) | rotation estimation · Point cloud alignment | test performance | Helped |
| use distance picking for quaternions (ablation) | rotation estimation · Point cloud alignment | test performance | Hurt |
| add a half-space map to quaternion inputs (ablation) | cube image prediction from rotation · Cube rotation to images | pixel-wise MSE | Helped |
| use R9+SVD instead of R6+GSO | feature prediction · Cube rotation to images | pixel-wise MSE | Helped |
| use Euler angles instead of R9+SVD or R6+GSO | 6D object pose estimation · YCB-Video | accuracy | Hurt |
| use quaternion representation instead of R9+SVD | 6D object pose estimation · YCB-Video | accuracy | No clear effect |
| add quaternion augmentation (ablation) | Fourier-series feature prediction · SO(3) input, n_b=1 to n_b=3 | MSE | Helped |
| add quaternion augmentation (ablation) | Fourier-series feature prediction · SO(3) input, n_b=4 to n_b=5 | MSE | No clear effect |
Surprising findings
- Changing the loss (distance picking or distances computed in SO(3)) does not fix discontinuities caused by double cover.
- Directly predicting rotation matrix entries performs worse than Procrustes-based SVD/GSO methods.
Limitations
- Analysis focuses on regression with gradient-based training and a limited set of representations.
- Experiments are on relatively small-scale synthetic and benchmark tasks.
- Some claims (e.g., half-space maps for small rotations) rely on empirical evidence rather than proofs.
Experimental setup
Datasets: ModelNet airplane CAD models YCB-Video CMU MoCap
Benchmarks: YCB-Video
Metrics: chordal distance geodesic distance ADD-S AUC ADD-S <2cm accuracy pixel-wise MSE
Baselines: Euler angles quaternions exponential coordinates axis-angle GSO SVD
Builds on: 6D rotation representation (Zhou et al.) PoseCNN Procrustes problem
Closest papers by meaning
- On the Continuity of Rotation Representations in Neural Networks (CVPR 2019 · similarity 0.90)
- Self-Supervised Representation Learning by Rotation Feature Decoupling (CVPR 2019 · similarity 0.88)
- The Devil Is in the Pose: Ambiguity-Free 3D Rotation-Invariant Learning via Pose-Aware Convolution (CVPR 2022 · similarity 0.86)
- Unsupervised Representation Learning by Predicting Image Rotations (ICLR 2018 · similarity 0.86)
- CRIN: Rotation-Invariant Point Cloud Analysis and Rotation Estimation via Centrifugal Reference Frame (AAAI 2023 · similarity 0.86)
- Understanding Adam Requires Better Rotation Dependent Assumptions (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.