Research map / 3D vision / 3D generation
Prof. Robot: Differentiable Robot Rendering Without Static and Self-Collisions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Prof. Robot adds a learned, Eikonal-regularized collision classifier over joint angles to Dr. Robot's differentiable Gaussian-splatting robot rendering, so gradient-based pose optimization avoids static and self-collisions.
Problem
Differentiable robot rendering learns robot actions from image supervision but has no physical awareness, so optimized joint poses can cause collisions with static environment objects or the robot itself.
Approach
Train an MLP collision classifier over joint angles with a hierarchical (kinematic-tree) encoding and an Eikonal regularizer on the pre-sigmoid output, making it an approximate signed distance field whose gradients stay consistent for pose optimization; the classifier plugs into Dr. Robot's rendering loss as an extra penalty.
Key results
- The hierarchical MLP SDF classifier reaches 97.70% collision classification accuracy, versus 54.33% for an NDF-based model trained on approximate distance labels.
- Without the Eikonal loss, pose optimization needs about 1000 iterations with 99% collision rate; with it, about 120 iterations and 17% collision rate.
- Integrated into Dr. Robot for inverse action from images, collision rate drops from 78% to 9% on the first collision-point setting and to 2% in other settings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add collision SDF loss to Dr. Robot | Dr. Robot · Inverse action from collision image | collision rate: 78% -> 9% | Helped |
| Add SDF collision constraint | Dr. Robot trajectory optimization · Inverse trajectory optimization | collision rate: 10.64% -> 0.00% | Helped |
| Add collision SDF constraint | Dr. Robot VLM pose optimization · VLM pose optimization | dot product mean: 33.86 -> 34.22 | No clear effect |
| Replace hierarchical encoding with flattened encoding (ablation) | Prof. Robot · Pose collision classification | classification accuracy: 97.70% -> 97.51% | Hurt |
| Remove Eikonal regularizer (ablation) | Prof. Robot · Collision resolution | collision rate: 17% -> 99% | Hurt |
| Remove Eikonal regularizer (ablation) | Prof. Robot · Pose collision classification | classification accuracy: 97.70% -> 98.39% | Helped |
| Remove scaling factor s (ablation) | Prof. Robot · Pose collision classification | classification accuracy: 97.70% -> 85.37% | Hurt |
| Replace SDF modeling with NDF modeling (ablation) | Prof. Robot · Pose collision classification | classification accuracy: 97.70% -> 54.33% | Hurt |
| Set collision-resolution SDF threshold from −0.000 to −0.100 (ablation) | Prof. Robot pose optimization · Collision resolution | collision points after optimization: 0.399 -> 0.001 | Helped |
| Vary the pose-optimization learning rate across 0.001–1.000 (ablation) | Prof. Robot pose optimization · Collision resolution | collision points after optimization: 0.003; 0.001; 0.000; 0.000; 0.000; 0.001; 0.000; 0.000 | No clear effect |
Surprising findings
- Without the Eikonal regularizer the gradient norm collapses to about 1.9e-6, making the classifier nearly useless for gradient-based optimization.
- Classification learning alone yields high accuracy (98.39% without Eikonal) but poor optimization behavior, showing accuracy does not reflect gradient quality.
Limitations
- Only static environments are modeled; cross-environment generalization and dynamic objects are not handled.
- Evaluated on a small number of scenes (six) and a limited set of robot setups.
Experimental setup
Datasets: MuJoCo scenes from Dr. Robot configurations (six scenes, sampled poses)
Benchmarks: MuJoCo-simulated collision scenes Dr. Robot inverse action from images CLIP shadow-hand action optimization sim-to-real obstacle avoidance
Metrics: classification accuracy Eikonal loss collision point count collision rate iteration count dot product
Baselines: Dr. Robot NDF-based collision-free manifold
Builds on: Dr. Robot Gaussian splatting Eikonal regularization Neural Distance Field collision-free manifold (Yuan et al.)
Closest papers by meaning
- DynaRend: Learning 3D Dynamics via Masked Future Rendering for Robotic Manipulation (NeurIPS 2025 · similarity 0.86)
- Robots Pre-train Robots: Manipulation-Centric Robotic Representation from Large-Scale Robot Datasets (ICLR 2025 · similarity 0.85)
- Physics-informed Temporal Difference Metric Learning for Robot Motion Planning (ICLR 2025 · similarity 0.84)
- DynaFLIP: Rethinking Robotics Perception via Tri-Modal-Dynamics Guided Representation (NeurIPS 2026 · similarity 0.84)
- DiffTORI: Differentiable Trajectory Optimization for Deep Reinforcement and Imitation Learning (NeurIPS 2024 · similarity 0.84)
- Localizing, Structuring, and Rendering: Bridging 3D and 2D Vision-Language-Action Models for Robotic Manipulation (CVPR 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.