Research map / 3D vision / Gaussian splatting
PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PhysGaussian, which evolves 3D Gaussian kernels with a custom MPM simulator and continuum mechanics so that the same Gaussians serve for both physical simulation and rendering.
Problem
Generating novel physics-based dynamics for NeRF/3D Gaussian scenes usually requires meshing, tetrahedralization, or cage/proxy meshes, creating a mismatch between simulation and rendering.
Approach
Treat 3D Gaussian kernels directly as MPM particles: deform each kernel's center and covariance via the deformation gradient from continuum mechanics, rotate spherical harmonics accordingly, and splat the deformed kernels for rendering, so simulation and rendering share one representation.
Key results
- On the lattice deformation benchmark, the method achieves PSNR up to 31.15 (stool bend) and outperforms NeRF-Editing, Deforming-NeRF and PAC-NeRF in all six test cases.
- Ablations show that fixed covariance (no kernel deformation) and fixed harmonics each reduce PSNR relative to the full method (e.g., plant twist 23.09 and 23.69 vs 23.87).
- Real-time simulation plus rendering is reported for simple scenes: plane 30 FPS, toast 25 FPS, jam 36 FPS on an RTX 3090.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use deformable covariance instead of translating Gaussian kernels only (ablation) | PhysGaussian · Wolf, Stool, and Plant bend/twist lattice benchmarks | PSNR: Fixed Covariance: 26.77, 26.02, 29.94, 25.31, 23.95, 23.09; Ours: 26.96, 26.46, 31.15, 26.15, 25.81, 23.87 | Helped |
| Use deformable covariance instead of rigid transformations of Gaussian kernels only (ablation) | PhysGaussian · Wolf, Stool, and Plant bend/twist lattice benchmarks | PSNR: Rigid Covariance: 26.84, 26.16, 30.28, 25.70, 24.09, 23.53; Ours: 26.96, 26.46, 31.15, 26.15, 25.81, 23.87 | Helped |
| Rotate spherical-harmonic orientations with the deformation (ablation) | PhysGaussian · Wolf, Stool, and Plant bend/twist lattice benchmarks | PSNR: Fixed Harmonics: 26.83, 26.02, 30.87, 25.75, 25.09, 23.69; Ours: 26.96, 26.46, 31.15, 26.15, 25.81, 23.87 | Helped |
| Use PhysGaussian instead of the strongest listed baseline | PhysGaussian · Wolf bend, Wolf twist, Stool bend, Stool twist, Plant bend, Plant twist | PSNR: Ours: 26.96, 26.46, 31.15, 26.15, 25.81, 23.87; strongest baselines: 26.91, 25.27, 22.32, 21.26, 19.85, 18.63 | Helped |
Surprising findings
- Rigid-covariance variant (as in prior work) is competitive with the full method on several cases, suggesting covariance deformation matters less than other components in some scenes.
Limitations
- Shadow evolution is not modeled and material parameters are manually set.
- Only handles solid/viscoplastic/granular materials; liquids are left to future work.
- Internal filling relies on heuristic opacity-grid ray casting rather than a learned model.
Experimental setup
Datasets: sofa suite (BlenderNeRF synthetic) fox (Instant-NGP) plane (Nerfstudio) ruins (DroneDeploy NeRF) toast (real iPhone capture) jam (real iPhone capture) lattice deformation benchmark (BlenderNeRF)
Benchmarks: lattice deformation benchmark (wolf, stool, plant; bend and twist)
Metrics: PSNR
Baselines: NeRF-Editing Deforming-NeRF PAC-NeRF
Builds on: 3D Gaussian Splatting Material Point Method Stomakhin et al. MPM snow PAC-NeRF Zong et al. MPM
Closest papers by meaning
- OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation (ICLR 2025 · similarity 0.90)
- PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation (NeurIPS 2025 · similarity 0.88)
- PhysX-3D: Physical-Grounded 3D Asset Generation (NeurIPS 2025 · similarity 0.87)
- Gaussian Splashing: Unified Particles for Versatile Motion Synthesis and Rendering (CVPR 2025 · similarity 0.87)
- DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors (AAAI 2025 · similarity 0.86)
- DecoupledGaussian: Object-Scene Decoupling for Physics-Based Interaction (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.