Research map / Video / Video segmentation and tracking

DeformMaster: An Interactive Physics-Neural World Model for Deformable Objects from Videos

NeurIPS 2026 · Video › Video segmentation and tracking · approach: Physics-inspired motion prediction · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds DeformMaster, a video-derived interactive world model for deformable objects that combines a neural-residual particle-grid MPM simulator, compliant distributed hand actuators, mixture-of-constitutive-experts materials, and LBS-driven Gaussian splatting for rendering.

Problem

Real-world deformable-object world models need to recover physical state from monocular or multi-view videos and roll it forward stably under novel hand actions while rendering high-fidelity appearance from novel views.

Approach

Pairs a differentiable MPM physics rollout with a bounded neural particle-grid velocity residual, models hand interaction as compliant distributed actuator-particle couplings, represents materials as a spatially varying mixture of hyperelastic experts, and drives Gaussian splats from the predicted particle trajectory.

Key results

• Future dynamics on 20 PhysTwin sequences: IoU 0.748, Chamfer 0.011, Track 0.024 vs PhysTwin 0.734/0.012/0.023; Spring-Gaus and GS-Dynamics have Chamfer 0.062 and 0.041.
• Appearance: PSNR 25.41, SSIM 0.936, LPIPS 0.061, matching or slightly beating PhysTwin (25.16/0.935/0.061).
• Ablations: removing the neural residual (MPM only) drops IoU to 0.656 and raises Track error to 3.94e-2; rigid DCA diverges on 9/20 sequences, and a single Neo-Hookean expert lowers IoU to 0.658.

Surprising findings

• Rigid (hard-constraint) actuator coupling diverged on 9 of 20 sequences, including all 8 volumetric cases.
• On planar cloth sequences DeformMaster's Track error (0.032) is worse than PhysTwin (0.028), the only category where it loses on Track.

Limitations

• Particle-grid simulation is less suited to very thin planar objects than spring-mass models with explicit topology, leaving a planar Track gap to PhysTwin.
• Interaction modeling depends on observed hand motion; severe occlusion degrades action-conditioned rollout.
• Runtime depends on MPM substep count, creating a fidelity-runtime trade-off.

Experimental setup

Datasets: PhysTwin sequences

Benchmarks: PhysTwin 20-sequence benchmark

Metrics: IoU Chamfer distance Track error PSNR SSIM LPIPS

Baselines: PhysTwin Spring-Gaus GS-Dynamics

Builds on: PhysTwin PhysGaussian Particle-Grid Neural Dynamics CoTracker3 TRELLIS

Closest papers by meaning

• DeformGen: Dynamics-Based Topology Augmentation for Deformable Manipulation Policy Learning (NeurIPS 2026 · similarity 0.88)
• RoboScape: Physics-informed Embodied World Model (NeurIPS 2025 · similarity 0.86)
• InterDyn: Controllable Interactive Dynamics with Video Diffusion Models (CVPR 2025 · similarity 0.86)
• DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors (AAAI 2025 · similarity 0.86)
• PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation (NeurIPS 2025 · similarity 0.86)
• MonoPhysics: Estimating Geometry, Appearance, and Physical Parameters from Monocular Videos (NeurIPS 2026 · similarity 0.86)