Research map / 3D vision / 3D generation
gradSim: Differentiable simulation for system identification and visuomotor control
J. Krishna Murthy, Miles Macklin, Florian Golemo, Vikram Voleti, Linda Petrini, Martin Weiss, Breandan Considine, Jérôme Parent-Lévesque, Kevin Xie, Kenny Erleben, Liam Paull, Florian Shkurti and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| differentiate through physics and rendering to estimate mass from video | ∇Sim · rigid mass estimation | Abs. Rel. err.: 9.01e-5 vs 0.0094 | Improved |
| use image-space video supervision instead of precise 3D state supervision | rigid mass estimation · rigid mass estimation | Abs. Rel. err.: 9.01e-5 vs 5.17e-9 | Hurt |
| estimate contact parameters from video using the unified simulator | ∇Sim · rigid-body parameter estimation | friction mu Rel. MAE: 0.0107 -> 0.0073 | Improved |
| estimate cloth velocity from images rather than 3D state labels | ∇Sim · cloth parameter estimation | per-particle velocity Rel. MAE: 0.127 -> 0.026 | Improved |
| omit friction from the dynamics model (ablation) | ∇Sim · mass estimation with imperfect simulator models | Mean Rel. Abs. Err.: 0.1071 -> 0.1866 | Hurt |
| omit elasticity from the dynamics model (ablation) | ∇Sim · mass estimation with imperfect simulator models | Mean Rel. Abs. Err.: 0.1071 -> 0.2281 | Hurt |
| model rigid objects as deformable (ablation) | ∇Sim · mass estimation with imperfect simulator models | Mean Rel. Abs. Err.: 0.1071 -> 0.3462 | Hurt |
| model deformable objects as rigid (ablation) | ∇Sim · mass estimation with imperfect simulator models | Mean Rel. Abs. Err.: 0.1071 -> 0.4974 | Hurt |
| replace differentiable rasterization with photorealistic rendering (ablation) | ∇Sim · mass estimation with imperfect simulator models | Mean Rel. Abs. Err.: 0.1071 -> 0.1793 | Hurt |
| include shading and texture cues instead of using only uniform color (ablation) | ∇Sim renderer · rigid mass estimation | convergence speed | Improved |
| use texture rather than shaded-only rendering (ablation) | ∇Sim renderer · rigid mass estimation | convergence speed | Improved |
| use a single visual target image rather than dense 3D supervision | ∇Sim visuomotor control · control-fem | goal attainment | Mixed |
Closest papers by meaning
- RISP: Rendering-Invariant State Predictor with Differentiable Simulation and Rendering for Cross-Domain Parameter Estimation (ICLR 2022 · similarity 0.88)
- Physics-as-Inverse-Graphics: Unsupervised Physical Parameter Estimation from Video (ICLR 2020 · similarity 0.86)
- Toward Material-Agnostic System Identification from Videos (ICCV 2025 · similarity 0.86)
- Vid2Sim: Generalizable, Video-based Reconstruction of Appearance, Geometry and Physics for Mesh-free Simulation (CVPR 2025 · similarity 0.85)
- DeformMaster: An Interactive Physics-Neural World Model for Deformable Objects from Videos (NeurIPS 2026 · similarity 0.85)
- DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
