Research map / 3D vision / Human pose estimation

3D Equivariant Pose Regression via Direct Wigner-D Harmonics Prediction

NeurIPS 2024 · 3D vision › Human pose estimation · approach: Learned camera pose estimation · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Predicts Wigner-D harmonic coefficients directly with an SO(3)-equivariant spherical CNN and trains with a frequency-domain MSE loss for single-image 3D rotation regression.

Problem

Single-image 3D orientation estimation suffers from discontinuities and singularities of spatial rotation parameterizations (Euler angles, quaternions, etc.), and spherical CNN-based SO(3)-equivariant models operate in the frequency domain where these parameterizations are incompatible.

Approach

Regress Wigner-D coefficients directly in the frequency domain with an S2/SO(3) equivariant spherical CNN head, train with a frequency-domain MSE loss, and decode rotations at inference by querying a HEALPix-based SO(3) grid with a softmax over similarities.

Key results

• On ModelNet10-SO(3) with ResNet-101, achieves Acc@15 0.773 and median error 11.9°, versus 12.2° for the best prior median (Liu et al. Fisher).
• On PASCAL3D+ with ResNet-101, achieves Acc@30 0.892 and median error 8.6°, versus 9.2° for Howell et al.
• In ablation, Wigner-D output (Acc@15 0.6807, median 22.27°) far outperforms Euler (132.56°), quaternion (75.95°), axis-angle, and rotation-matrix (37.54°) outputs.

Measured effects reported in the paper

Surprising findings

• On SYMSOL I, the Wigner-D regression loss alone performs worse than the distribution-loss baseline for symmetric objects, and sharp predicted distributions hurt average log likelihood.
• Replacing equivariant layers with conventional convolutions drastically degrades performance (Acc@15 0.1056 vs 0.6807).

Limitations

• Single-pose regression assumes unambiguous poses; symmetric objects need an added distribution loss.
• Sharp predicted distributions yield low log-likelihood scores on SYMSOL II.
• Frequency truncation at L=6 limits precision; geodesic loss in the frequency domain is ineffective.

Experimental setup

Datasets: ModelNet10-SO(3) PASCAL3D+ SYMSOL I SYMSOL II

Benchmarks: ModelNet10-SO(3) PASCAL3D+ SYMSOL I SYMSOL II

Metrics: median rotation error Acc@15 Acc@30 Acc@3 Acc@5 Acc@10 average log likelihood

Baselines: Klee et al. (I2S) Liu et al. (Uni/Fisher) Yin et al. Murphy et al. Mohlin et al. Deng et al. Howell et al. Liao et al. Prokudin et al. Zhou et al.

Builds on: spherical CNNs image2sphere I2S I-PDF RotLaplace

Closest papers by meaning

• Image to Sphere: Learning Equivariant Features for Efficient Pose Prediction (ICLR 2023 · similarity 0.92)
• Leveraging Equivariant Features for Absolute Pose Regression (CVPR 2022 · similarity 0.90)
• Equivariant Single View Pose Prediction Via Induced and Restriction Representations (NeurIPS 2023 · similarity 0.89)
• $SE(3)$ Equivariant Ray Embeddings for Implicit Multi-View Depth Estimation (NeurIPS 2024 · similarity 0.88)
• A Functional Approach to Rotation Equivariant Non-Linearities for Tensor Field Networks. (CVPR 2021 · similarity 0.87)
• Pose-Transformed Equivariant Network for 3D Point Trajectory Prediction (CVPR 2024 · similarity 0.86)