Research map / 3D vision / 3D generation

Deep SE(3)-Equivariant Geometric Reasoning for Precise Placement Tasks

ICLR 2024 · 3D vision › 3D generation · approach: 2D-to-3D pretrained representations · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RelDist, an SE(3)-invariant pairwise distance representation over two objects, decoded into a provably SE(3)-equivariant cross-pose via differentiable multilateration and Procrustes (SVD) layers, trained end-to-end from few demonstrations.

Problem

Predicting precise relative placement poses (e.g., hanging a mug on a rack) from high-dimensional point cloud demonstrations, where the prediction must be SE(3)-equivariant to object poses and invariant to camera pose, and learning from few demonstrations is hard without inductive biases.

Approach

Predict an SE(3)-invariant pairwise distance matrix (RelDist) between points of object A and object B, recover each A-point's goal location by closed-form differentiable multilateration, then fit the rigid transform with differentiable weighted Procrustes (SVD); the composition is provably equivariant to transforms of either object.

Key results

• On RLBench placement tasks, the method achieves rotation errors below 1.25 degrees and translation errors below 3 mm, and is 2-9x more precise in rotation than TAX-Pose on 4 of 5 tasks.
• On NDF mug hanging (upright), overall success at 1cm penetration is 0.35 for the method vs 0.15 for TAX-Pose and 0.99 at infinite threshold vs 0.93.
• With a single demonstration on stack_wine, the method still achieves very high precision, with higher sample efficiency than TAX-Pose.

Measured effects reported in the paper

Surprising findings

• Retrained TAX-Pose and the proposed method both performed substantially worse than originally reported TAX-Pose results on NDF bottle and bowl tasks, attributed to missing symmetry-breaking code, so those results were inconclusive.
• On bowl and bottle NDF tasks, the proposed method underperforms TAX-Pose at the 1cm/3cm penetration thresholds, and NDF has higher infinity-threshold success.

Limitations

• Cannot handle symmetric objects or multimodal placements without an external symmetry-breaking mechanism, since only a single pose is predicted.
• Assumes the two task-relevant objects are already segmented.
• Kernel matrix scales as N^2 in point count, requiring subsampling.

Experimental setup

Datasets: RLBench placement tasks (10 demos each) NDF mug hanging training data real-world mug-hanging demonstrations (from TAX-Pose authors)

Benchmarks: RLBench placement tasks (stack_wine, toilet_roll, hang_hanger, phone_on_base, insert_peg) NDF mug hanging NDF bottle placement NDF bowl placement real-world mug hanging offline dataset

Metrics: rotation error (degrees) translation error (m) grasp/place/overall success rate under penetration threshold

Baselines: TAX-Pose Neural Descriptor Fields Dense Object Nets

Builds on: TAX-Pose Vector Neurons DGCNN Neural Descriptor Fields Dense Object Nets

Closest papers by meaning

• ET-SEED: EFFICIENT TRAJECTORY-LEVEL SE(3) EQUIVARIANT DIFFUSION POLICY (ICLR 2025 · similarity 0.86)
• Hierarchical Equivariant Policy via Frame Transfer (ICML 2025 · similarity 0.85)
• Geometry-aware RL for Manipulation of Varying Shapes and Deformable Objects (ICLR 2025 · similarity 0.85)
• 3D Equivariant Visuomotor Policy Learning via Spherical Projection (NeurIPS 2025 · similarity 0.85)
• SE(3)-Equivariant Diffusion Policy in Spherical Fourier Space (ICML 2025 · similarity 0.84)
• GravMAD: Grounded Spatial Value Maps Guided Action Diffusion for Generalized 3D Manipulation (ICLR 2025 · similarity 0.84)