Research map / 3D vision / 3D generation
DecoupledGaussian: Object-Scene Decoupling for Physics-Based Interaction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DecoupledGaussian, a system that restores geometry of an object and its contacted scene from in-the-wild videos via joint Poisson fields and multi-view carving, enabling MLS-MPM simulations where objects detach under user impulses.
Problem
Physics-based simulation of Gaussian-splatting objects from in-the-wild captures is limited to elastic jitter on the contact surface because objects cannot be decoupled from their supporting scene and the occluded/missing geometry of both is unrestored.
Approach
Joint Poisson fields reconstruct indicator functions for object and scene jointly (resolving intersections), with proxy points from unbiased planar-GS depth as object input, and a unilateral negative cross-entropy carving loss using SAM2 negative masks; 2D inpainting (LaMa) is used only for texture refinement.
Key results
- Object restoration PSNR 30.32 vs 26.62 for GIC and 24.46 for PhysGaussian on the decoupling benchmark.
- Scene restoration PSNR 27.32 and CD 4.40e-3, outperforming VR-GS (25.13, 6.41e-3) and GScream (17.82, 44.00e-3).
- User study interactive simulation fidelity 4.35 vs 2.73 for the best baseline combination (GIC for object).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dense interior points PO (ablation) | DecoupledGaussian · Object-Scene Interaction Simulation | Motion-FID: 48.39 -> 31.69 | Helped |
| add proxy points PO (ablation) | DecoupledGaussian · Object-Scene Interaction Simulation | Motion-FID: 36.01 -> 31.69 | Helped |
| add joint Poisson fields W (ablation) | DecoupledGaussian · Object-Scene Interaction Simulation | Motion-FID: 42.27 -> 31.69 | Helped |
| apply UNCE multi-view carving (ablation) | DecoupledGaussian object restoration · Object restoration | geometry alignment | Helped |
| use planar-based Gaussian Splatting instead of vanilla Gaussian Splatting (ablation) | DecoupledGaussian · Object-Scene Interaction Simulation | geometry artifacts | Helped |
Surprising findings
- Object FID remains high (138.75) despite best PSNR, attributed to reliance on training-view interpolation for texture restoration.
- 2D inpainting tools (LaMa, PhotoRoom) produce artifacts and inconsistent textures across frames, motivating geometry-first restoration.
Limitations
- Evaluation does not cover complex scenes with multiple objects in varying contact configurations.
- High-frequency texture completion for object restoration is challenging; object FID is high.
- Decoupling fine-grained object components remains difficult.
Experimental setup
Datasets: Mip-NeRF360 Tanks&Temples Deep Blending LERF figurines Instruct-NeRF2NeRF bear PEGASET BlenderNeRF scenes
Benchmarks: Decoupling benchmark (PEGASET, BlenderNeRF scenes) Mip-NeRF360 Tanks&Temples Deep Blending
Metrics: PSNR LPIPS FID Chamfer Distance Motion-FID user study rating (1-5)
Baselines: PhysGaussian GIC VR-GS GScream
Builds on: 3D Gaussian Splatting PGSR PhysGaussian Poisson surface reconstruction MLS-MPM
Closest papers by meaning
- Motion Decoupled 3D Gaussian Splatting for Dynamic Object Representation (AAAI 2025 · similarity 0.87)
- Simulation-Ready Compositional 3D Scene Reconstruction from a Single Image (NeurIPS 2026 · similarity 0.87)
- MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.86)
- H3D-DGS: Exploring Heterogeneous 3D Motion Representation for Deformable 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.86)
- HybridGS: Decoupling Transients and Statics with 2D and 3D Gaussian Splatting (CVPR 2025 · similarity 0.86)
- OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.