Research map / 3D vision / 3D generation
OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OmniPhysGS, which represents each 3D Gaussian as a constitutive model chosen from 12 expert MPM material models and trains it with SDS from a pretrained video diffusion model to generate physics-based 4D dynamics for heterogeneous materials.
Problem
Existing physics-based 3D/4D dynamics synthesis from Gaussians assumes a single predefined material category (e.g., elasticity) and fixed constitutive models, failing to handle heterogeneous materials and multi-material interactions in one scene.
Approach
Each Gaussian neighborhood gets a learnable, physics-guided constitutive model: a PointNet-style encoder plus a decoder that predicts hard-max selections over 12 expert hyperelastic and plasticity models inside a differentiable MPM simulator, optimized end-to-end by SDS from ModelScope video diffusion.
Key results
- On single-object cases, average Diff_SSIM 0.0731 and CLIPSIM 20.716% vs. best baseline (DreamPhysics) 0.0307 and 19.588%.
- On multi-object scenes, average CLIPSIM 27.552% vs. 26.836% for DreamPhysics and Diff_CLIP 1.0569 vs. 1.0287.
- Their MPM solver reduces training memory from 48,073 MB (NCLaw) to 13,889 MB, saving about 75% GPU memory, though training time rises from 21.3 s to 47.8 s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use learnable Constitutive Gaussians instead of fixed constitutive models | 3D physics-based dynamics generation · Single object, different materials (average) | CLIPSIM: 19.634 -> 20.716 | Helped |
| Use learnable Constitutive Gaussians to model heterogeneous materials | 3D physics-based dynamics generation · Multiple objects, different materials (average) | CLIPSIM: 26.836 -> 27.552 | Helped |
| Add expert-designed physical constitutive models to the decoder (ablation) | OmniPhysGS · Single object, different materials (average) | Diff_SSIM: 0.0536 -> 0.0731 | Helped |
| Add expert-designed physical constitutive models to the decoder (ablation) | OmniPhysGS · Multiple objects, different materials (average) | Diff_SSIM: 0.0126 -> 0.0822 | Helped |
| Use multi-batch training (ablation) | OmniPhysGS · Single object, different materials (average) | Diff_SSIM: 0.0638 -> 0.0731 | Helped |
| Use multi-batch training (ablation) | OmniPhysGS · Multiple objects, different materials (average) | Diff_SSIM: 0.0444 -> 0.0822 | Helped |
| Group rendered frames into stages (ablation) | OmniPhysGS · Single and multiple objects, different materials | training feasibility | Helped |
| Replace NCLaw's MPM solver with the vectorized Torch implementation | MPM simulation · 5×10^4 particles and 1×10^3 time steps | Train Memory (MB): 48,073 -> 13,889 | Helped |
| Replace NCLaw's MPM solver with the vectorized Torch implementation | MPM simulation · 5×10^4 particles and 1×10^3 time steps | Train Time (s): 21.3 -> 47.8 | Hurt |
Surprising findings
- Baselines with fixed constitutive models produce nearly identical dynamics regardless of prompt, implying tuning physical parameters alone is insufficient.
- Training without grouping runs out of memory, and dropping the physical prior (plain MLP) sharply lowers multi-object Diff_SSIM (0.0126 vs 0.0822).
Limitations
- Limited to 12 expert-designed constitutive models covering a few representative materials.
- Per-scene SDS optimization is time-intensive.
- Training time per solver iteration is higher than NCLaw (47.8 s vs 21.3 s).
Experimental setup
Datasets: PhysGaussian dataset BlenderNerf-generated scenes
Benchmarks: PhysGaussian dataset scenes (custom single- and multi-object prompts)
Metrics: CLIPSIM Diff_SSIM Diff_CLIP
Baselines: PhysDreamer Physics3D DreamPhysics NCLaw w/o physical prior MLP
Builds on: PhysGaussian NCLaw DreamPhysics Physics3D PhysDreamer Score Distillation Sampling
Closest papers by meaning
- PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics (CVPR 2024 · similarity 0.90)
- Gaussian Splashing: Unified Particles for Versatile Motion Synthesis and Rendering (CVPR 2025 · similarity 0.89)
- PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation (NeurIPS 2025 · similarity 0.88)
- DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors (AAAI 2025 · similarity 0.88)
- MonoPhysics: Estimating Geometry, Appearance, and Physical Parameters from Monocular Videos (NeurIPS 2026 · similarity 0.88)
- Motion Decoupled 3D Gaussian Splatting for Dynamic Object Representation (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.