Research map / 3D vision / 3D generation
FreeGave: 3D Physics Learning from Dynamic Videos by Gaussian Velocity
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns 3D scene geometry, appearance and per-Gaussian velocity fields from multi-view videos via a physics code and divergence-free velocity parameterization, enabling future frame extrapolation and unsupervised motion segmentation without PINN losses or object priors.
Problem
Learning 3D physics (velocity) of complex dynamic scenes purely from multi-view RGB videos so that future frames can be extrapolated, without PINN losses or object masks/types.
Approach
Attach a latent physics code to each canonical 3D Gaussian and decode a divergence-free rigid-particle velocity field (six basis velocity components times a fixed basis matrix) via a bottleneck-weight factorization, trained with an auxiliary deformation field and an interleaved midpoint integrator instead of PINN losses.
Key results
- On Dynamic Object Dataset future extrapolation, FreeGave reaches 31.987 PSNR versus 27.594 for NVFi and 29.027 interpolation-free baseline NVFi comparison; best on all datasets.
- On the Dynamic Indoor Scene dataset segmentation, FreeGave attains 99.75 AP and 96.27 PQ versus 91.21 AP for NVFi and 65.37 AP for Mask2Former.
- On ParticleNeRF extrapolation FreeGave reaches 26.657 PSNR versus 22.730 for DefGSnvfi.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FreeGave instead of DefGSnvfi | Dynamic Object Dataset extrapolation · Dynamic Object Dataset extrapolation | PSNR: 28.749 -> 31.987 | Helped |
| use FreeGave instead of DefGSnvfi | Dynamic Indoor Scene Dataset extrapolation · Dynamic Indoor Scene Dataset extrapolation | PSNR: 31.096 -> 35.019 | Helped |
| use FreeGave instead of DefGSnvfi | ParticleNeRF Dataset extrapolation · ParticleNeRF Dataset extrapolation | PSNR: 22.730 -> 26.657 | Helped |
| use FreeGave instead of DefGSnvfi | FreeGave-GoPro Dataset extrapolation · FreeGave-GoPro Dataset extrapolation | PSNR: 26.431 -> 28.094 | Helped |
| use a position-conditioned physics code network instead of a learnable per-Gaussian code (ablation) | FreeGave · Dynamic Object Dataset extrapolation | PSNR: 25.961 -> 31.987 | Helped |
| add divergence-free basis parametrization (ablation) | FreeGave velocity field · Dynamic Object Dataset extrapolation | PSNR: 29.400 -> 31.987 | Helped |
| add bottleneck vector decomposition for velocity components (ablation) | FreeGave velocity field · Dynamic Object Dataset extrapolation | PSNR: 29.432 -> 31.987 | Helped |
| use 16 motion patterns instead of 8 (ablation) | FreeGave · Dynamic Object Dataset extrapolation | PSNR: 30.972 -> 31.987 | Helped |
| use 16 motion patterns instead of 32 (ablation) | FreeGave · Dynamic Object Dataset extrapolation | PSNR: 31.438 -> 31.987 | Helped |
| add the auxiliary deformation field (ablation) | FreeGave · Dynamic Object Dataset extrapolation | PSNR: 17.927 -> 31.987 | Helped |
| add physics code input to the deformation field (ablation) | FreeGave · Dynamic Object Dataset extrapolation | PSNR: 31.217 -> 31.987 | Helped |
| add scaling deformation to the deformation field (ablation) | FreeGave · Dynamic Object Dataset extrapolation | PSNR: 31.704 -> 31.987 | Helped |
Surprising findings
- Removing the auxiliary deformation field causes training to collapse on the Dynamic Indoor Scene dataset, making it the most influential ablation.
- The unsupervised physics-code grouping outperforms the fully supervised Mask2Former on AP, PQ, and mIoU for motion segmentation.
Limitations
- Baselines are mostly NeRF-based and the real-world dataset is small (6 scenes)
- Interleaved midpoint integration with an auxiliary deformation field is needed for stable training; direct integration is unstable under memory limits
- Real-world evaluation limited to self-collected GoPro scenes with fixed cameras
Experimental setup
Datasets: Dynamic Object Dataset Dynamic Indoor Scene Dataset ParticleNeRF Dataset FreeGave-GoPro Dataset
Benchmarks: Dynamic Object Dataset Dynamic Indoor Scene Dataset ParticleNeRF Dataset FreeGave-GoPro Dataset
Metrics: PSNR SSIM LPIPS AP PQ F1 mIoU
Baselines: T-NeRF D-NeRF NSFF TiNeuVox NVFi DefGS DefGSnvfi Mask2Former
Builds on: 3D Gaussian splatting Deformable 3D Gaussians NVFi physics-informed neural networks
Closest papers by meaning
- FreeTimeGS: Free Gaussian Primitives at Anytime Anywhere for Dynamic Scene Reconstruction (CVPR 2025 · similarity 0.88)
- SfM-Free 3D Gaussian Splatting via Hierarchical Training (CVPR 2025 · similarity 0.87)
- HiCoM: Hierarchical Coherent Motion for Dynamic Streamable Scenes with 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.87)
- Motion Decoupled 3D Gaussian Splatting for Dynamic Object Representation (AAAI 2025 · similarity 0.87)
- Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting (ICLR 2024 · similarity 0.87)
- DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.