Research map / Point clouds / Point cloud classification
Object Dynamics Modeling with Hierarchical Point Cloud-based Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a point-convolution U-Net (Object and Relational PointConv) that predicts object dynamics from point clouds and mesh vertices, outperforming GNN simulators on Physion and Kubric.
Problem
Learning object dynamics and collision reasoning in 3D from point clouds or meshes, where GNN-based simulators ignore geometry and struggle with gravity and long-range interactions.
Approach
A hierarchical U-Net built from continuous PointConv layers that alternates within-object and cross-object convolutions, plus a mesh face-to-face collision scheme that computes features at dynamically chosen interaction points on mesh faces via PointConv interpolation.
Key results
- On Physion, the PointConv U-Net (acc) reaches 84.4% average contact prediction accuracy vs. 79.4% for SGNN (rigid scenarios averaged across 7 scenarios).
- PointConv U-Net (vel) reaches 81.3% average on Physion, with a Drape gain of more than 15 points over SGNN (75.8 vs 60.6).
- On Kubric, PointConv U-Net (acc) with point clouds achieves 1.87 Euclidean error on Movi-A vs. 2.20 for the GNN U-Net baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace SGNN with Ours (acc) | Physion dynamics prediction · Physion Support | contact prediction accuracy (%): 71.2 ± 0.9 -> 83.6 ± 3.5 | Helped |
| Replace SGNN with Ours (vel) | Physion dynamics prediction · Physion Drop | contact prediction accuracy (%): 74.3 ± 1.0 -> 89.8 ± 0.3 | Helped |
| Replace SGNN with Ours (vel) | Physion dynamics prediction · Physion Drape | contact prediction accuracy (%): 60.6 ± 0.5 -> 75.8 ± 1.9 | Helped |
| Replace GNN layers with PointConv layers (ablation) | Physion U-Net (velocity prediction) · Physion Contain | contact prediction accuracy (%): 69.3 ± 0.9 -> 82.4 ± 2.8 | Helped |
| Replace GNN layers with PointConv layers (ablation) | Physion U-Net (velocity prediction) · Physion Support | contact prediction accuracy (%): 62.7 ± 3.8 -> 76.2 ± 2.1 | Helped |
| Replace GNN layers with PointConv layers (ablation) | Physion U-Net (acceleration prediction) · Physion Support | contact prediction accuracy (%): 59.1 ± 1.1 -> 83.6 ± 3.5 | Helped |
| Enable explicit face reasoning on mesh input (ablation) | Ours (vel) · Kubric Movi-A | Euclidean distance: 2.95 ± 0.08 -> 2.47 ± 0.04 | Helped |
| Enable explicit face reasoning on mesh input (ablation) | Ours (vel) · Kubric Movi-C | Euclidean distance: 2.70 ± 0.05 -> 2.63 ± 0.05 | Hurt |
| Enable explicit face reasoning on mesh input (ablation) | Ours (acc) · Kubric Movi-A | Euclidean distance: 2.49 ± 0.02 -> 1.98 ± 0.03 | Helped |
| Enable explicit face reasoning on mesh input (ablation) | Ours (acc) · Kubric Movi-C | Euclidean distance: 2.23 ± 0.01 -> 2.23 ± 0.02 | No clear effect |
| Use point-cloud input instead of mesh vertices (ablation) | Ours (vel) · Kubric Movi-A | Euclidean distance: 2.95 ± 0.08 -> 2.15 ± 0.12 | Helped |
| Use point-cloud input instead of mesh vertices (ablation) | Ours (acc) · Kubric Movi-C | Euclidean distance: 2.23 ± 0.02 -> 2.18 ± 0.01 | Helped |
Surprising findings
- Explicit face-to-face collision reasoning on meshes did not improve performance on Movi-C, where dense vertices already capture proximity.
- The velocity-based model underperforms SGNN on the Link scenario, attributed to label noise in Link training data.
Limitations
- Velocity-based model is more sensitive to label noise (weaker on Link).
- Explicit face reasoning gives no gain on dense meshes (Movi-C).
- Could not compare against FIGNet on Kubric because its generated sequences and code were unavailable.
Experimental setup
Datasets: Physion Kubric (Movi-A, Movi-C)
Benchmarks: Physion Kubric
Metrics: contact prediction accuracy Euclidean distance between predicted and ground-truth point
Baselines: GNS DPI SGNN GNN U-Net
Builds on: PointConv SGNN DPI-Net FIGNet
Closest papers by meaning
- Learning rigid-body simulators over implicit shapes for large-scale scenes and vision (NeurIPS 2024 · similarity 0.84)
- Learning Flexible Body Collision Dynamics with Hierarchical Contact Mesh Transformer (ICLR 2024 · similarity 0.84)
- HEIR: Learning Graph-Based Motion Hierarchies (NeurIPS 2025 · similarity 0.84)
- LinNet: Linear Network for Efficient Point Cloud Representation Learning (NeurIPS 2024 · similarity 0.84)
- Degrees of Freedom Matter: Inferring Dynamics from Point Trajectories (CVPR 2024 · similarity 0.83)
- 4Deform: Neural Surface Deformation for Robust Shape Interpolation (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.