Research map / Probabilistic and scientific modeling / Dynamical systems
Latent Intuitive Physics: Learning to Transfer Hidden Physics from A 3D Video
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes latent intuitive physics, a probabilistic particle fluid simulator whose learned latent prior is adapted from visual posteriors inferred via a neural renderer, enabling transfer of hidden fluid properties from a single 3D video to novel scenes.
Problem
Predicting and simulating fluid dynamics in novel scenes when hidden physical properties (e.g., viscosity, density) are unknown and only a single multi-view video of the fluid is available.
Approach
Learn a probabilistic particle simulator with per-particle latent features and a GRU-based prior learner; infer per-particle visual posterior latents by backpropagating photometric error through a frozen transition model and neural renderer, then finetune only the prior learner to match these posteriors so it can be plugged into the simulator for novel scenes.
Key results
- On novel geometry with rho=2000, nu=0.065, the proposed method reaches d-bar 34.54 vs. 51.10 for PAC-NeRF and 52.49 for CConv.
- On the particle dataset, the model achieves d_t+1 = 0.31 and d-bar = 38.37, beating DMCF (39.70) and CConv (44.79) in long-term error.
- On unseen dynamics (mixed fluids), it reaches 44.25 on unseen scenes vs. 46.83 for CConv and 90.51 for a global-latent variant.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| transfer visual posterior latents to the physical prior learner (ablation) | Latent Intuitive Physics · Unseen fluid geometries | average prediction error: 34.54±0.55 vs 42.43 | Helped |
| transfer visual posterior latents to the physical prior learner (ablation) | Latent Intuitive Physics · Unseen fluid geometries, ρ=1000, ν=0.08 | average prediction error: 33.11±0.50 vs 46.98 | Helped |
| transfer visual posterior latents to the physical prior learner (ablation) | Latent Intuitive Physics · Unseen fluid geometries, ρ=500, ν=0.2 | average prediction error: 39.03±0.79 vs 41.40 | Helped |
| adapt the physical prior learner using Stage C (ablation) | Latent Intuitive Physics · Novel fluid geometries | simulation availability | Helped |
| use estimated rather than true initial states for visual inference (ablation) | Latent Intuitive Physics · Observed and unseen fluid geometries | average prediction error: 32.41 vs 28.99; 34.54 vs 33.34; 32.97 vs 31.87; 33.11 vs 38.84; 41.15 vs 39.70; 39.03 vs 38.41 | Mixed |
| use particle-dependent, time-varying latent features instead of a global latent (ablation) | Latent Intuitive Physics · Generalization to unseen dynamics | average prediction error: 36.03 ± 0.80 vs 60.63; 44.25 ± 1.36 vs 90.51 | Helped |
| use Latent Intuitive Physics instead of the strongest baseline | Novel scene simulation · Unseen fluid geometries and boundary conditions | average prediction error: 34.54±0.55 vs 51.10; 39.86±0.90 vs 59.61; 33.11±0.50 vs 51.33; 37.79±0.84 vs 56.33; 39.03±0.79 vs 40.67; 47.25±1.26 vs 5 | Helped |
| use Latent Intuitive Physics instead of the strongest baseline | Observed Cuboid future prediction · ρ=2000, ν=0.065 and ρ=1000, ν=0.08 | average future prediction error: 32.41±0.17 vs 37.25; 32.97±0.71 vs 36.28 | Helped |
| use Latent Intuitive Physics instead of NeuroFluid | Observed Cuboid future prediction · ρ=500, ν=0.2 | average future prediction error: 41.15±0.71 vs 33.22 | Hurt |
| use a probabilistic simulator with latent features instead of the strongest deterministic particle baseline | Particle dataset · Short-term and full-sequence prediction | prediction error: 0.31±0.003 vs 0.34; 0.94±0.011 vs 1.03; 38.37±0.860 vs 39.70 | Helped |
Surprising findings
- NeuroFluid slightly outperforms the proposed method in future prediction on the observed scene at rho=500, nu=0.2 (33.22 vs 41.15), and PAC-NeRF beats the learned simulators on some geometries despite overfitting.
- Sys-ID with explicit physical parameters performs worst on novel scenes (e.g., 156.59 on geometry at rho=2000), suggesting explicit parameter inference transfers poorly.
Limitations
- Real-world validation is only a preliminary demonstration on static dyed-water images; dynamic real-world scenes are left to future work.
- Evaluation is on synthetic data with a single visual observation sequence per property set.
- Uses a learned neural renderer and voxel-based initial state estimation, adding pipeline complexity; the global-latent variant and Stage B/C ablations show sensitivity to design choices.
Experimental setup
Datasets: DFSPH particle dataset (SPlisHSPlasH generated, 600 scenes) Blender-rendered multi-view Cuboid fluid video (20 views)
Benchmarks: Novel scene simulation (unseen geometries and boundaries) Future prediction on observed Cuboid scene DFSPH particle dataset long-term prediction
Metrics: average Euclidean distance to closest predicted particle (d- one-to-one particle error (d_t+1, d_t+2)
Baselines: CConv NeuroFluid PAC-NeRF System Identification (Sys-ID) DPI-Net DMCF TIE Global Latent variant
Builds on: NeuroFluid PAC-NeRF CConv PhysNeRF NeRF
Closest papers by meaning
- Learning 3D Particle-based Simulators from RGB-D Videos (ICLR 2024 · similarity 0.89)
- DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors (AAAI 2025 · similarity 0.88)
- FreeGave: 3D Physics Learning from Dynamic Videos by Gaussian Velocity (CVPR 2025 · similarity 0.86)
- DEL: Discrete Element Learner for Learning 3D Particle Dynamics with Neural Rendering (NeurIPS 2024 · similarity 0.86)
- VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Models (NeurIPS 2025 · similarity 0.86)
- Lost in Latent Space: An Empirical Study of Latent Diffusion Models for Physics Emulation (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.