Research map / 3D vision / Face and shape
An Intuitive Multi-Frequency Feature Representation for SO(3)-Equivariant Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FER, a frequency-based SO(3)-equivariant feature representation built from a higher-dimensional rotation map D: SO(3)→SO(n), used as input to Vector Neurons to capture multi-frequency 3D detail.
Problem
Vector Neurons (VN) equivariant networks are limited because features are confined to 3D, which cannot capture the multi-frequency details of 3D shapes.
Approach
Map each point to an n-dimensional equivariant feature via a Lie-algebra-based representation D(R)=exp(θŵ·J) of rotations in SO(n), so the feature consists of sinusoids whose maximum frequency is floor((n-1)/2)/2π, and feed these features into VN networks.
Key results
- FER-VN-DGCNN reaches 90.5% z/z, 90.5% z/SO(3) and 90.5% SO(3)/SO(3) on ModelNet40 classification, the best among rotation-equivariant methods in the z/SO(3) and SO(3)/SO(3) settings.
- On volumetric ShapeNet reconstruction, FER-VN-OccNet reaches 71.9 mIoU across I/I, I/SO(3) and SO(3)/SO(3) settings, versus 69.3 for VN-OccNet at I/SO(3).
- FER-VN-DGCNN obtains the lowest normal-estimation error on ShapeNet (0.143 I/SO(3)) and ModelNet40 (0.078 I/SO(3)).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FER features with VN instead of 3D coordinate features | OccNet for shape reconstruction · ShapeNet reconstruction, I/I | volumetric mIoU: 69.3 -> 71.9 | Helped |
| Use FER features with VN instead of 3D coordinate features | OccNet for shape reconstruction · ShapeNet reconstruction, I/SO(3) | volumetric mIoU: 69.3 -> 71.9 | Helped |
| Use FER features with VN instead of 3D coordinate features | OccNet for shape reconstruction · ShapeNet reconstruction, SO(3)/SO(3) | volumetric mIoU: 68.8 -> 71.9 | Helped |
| Use FER features with VN instead of 3D coordinate features | PointNet normal estimation · ShapeNet | normal estimation error: 0.216 -> 0.196 | Helped |
| Use FER features with VN instead of 3D coordinate features | DGCNN normal estimation · ShapeNet | normal estimation error: 0.152 -> 0.143 | Helped |
| Use FER features with VN instead of 3D coordinate features | PointNet normal estimation · ModelNet40 | normal estimation error: 0.150 -> 0.125 | Helped |
| Use FER features with VN instead of 3D coordinate features | DGCNN normal estimation · ModelNet40 | normal estimation error: 0.092 -> 0.078 | Helped |
| Use FER features with VN instead of 3D coordinate features | PointNet classification · ModelNet40, z/SO(3) | accuracy: 77.5 -> 87.8 | Helped |
| Use FER features with VN instead of 3D coordinate features | DGCNN classification · ModelNet40, z/SO(3) | accuracy: 89.5 -> 90.5 | Helped |
| Use FER features with VN instead of 3D coordinate features | PointNet part segmentation · ShapeNet, z/SO(3) | average category mIoU: 72.4 -> 82.7 | Helped |
| Use FER features with VN instead of 3D coordinate features | DGCNN part segmentation · ShapeNet, SO(3)/SO(3) | average category mIoU: 81.4 -> 83.5 | Helped |
| Use FER features with VN instead of 3D coordinate features | FER-VN-DGCNN classification · ModelNet40, z/SO(3) | accuracy: 90.5 vs 91.4 | Mixed |
Surprising findings
- PaRINet, a rotation-invariant method, still outperforms FER-VN on ModelNet40 classification and ShapeNet segmentation in the z/SO(3) and SO(3)/SO(3) settings.
Limitations
- Constructing J1, J2 requires a non-linear optimization (Cross-Entropy Method) per choice of n.
- The feature is equivariant only for rotations about the chosen axis basis; the choice of e-hat depends on the z-axis reference.
- Classification and segmentation results do not beat the rotation-invariant PaRINet.
Experimental setup
Datasets: ShapeNet ModelNet40 EGAD
Benchmarks: ModelNet40 ShapeNet EGAD
Metrics: classification accuracy mean IoU normal estimation error 1-(n^T n_hat)^2 Chamfer Distance
Baselines: VN-PointNet VN-DGCNN VN-OccNet OccNet FA-PointNet FA-DGCNN DeepGMR ICP PaRINet TFN SVNet-DGCNN
Builds on: Vector Neurons NeRF PointNet DGCNN OccNet
Closest papers by meaning
- A Characterization Theorem for Equivariant Networks with Point-wise Activations (ICLR 2024 · similarity 0.85)
- Interpreting Equivariant Representations (ICML 2024 · similarity 0.85)
- 3D Equivariant Pose Regression via Direct Wigner-D Harmonics Prediction (NeurIPS 2024 · similarity 0.85)
- Pose-Transformed Equivariant Network for 3D Point Trajectory Prediction (CVPR 2024 · similarity 0.85)
- On the Fourier analysis in the SO(3) space : the EquiLoPO Network (ICLR 2025 · similarity 0.84)
- Learning 3D Equivariant Implicit Function with Patch-Level Pose-Invariant Representation (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.