Research map / 3D vision / 3D generation
FluidNexus: 3D Fluid Reconstruction and Prediction from a Single Video
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reconstructs and predicts 3D fluid appearance and velocity from a single video by synthesizing multi-view reference videos with Zero123 and CogVideoX refinement, then fitting a two-layer PBF-physics plus Gaussian-splatting particle model.
Problem
Single-video 3D fluid reconstruction and future prediction, where existing physics-informed neural fluid methods require multi-view synchronized videos.
Approach
Synthesize multiple novel-view videos from one input video using frame-wise novel-view diffusion (Zero123) plus SDEdit-style video diffusion refinement (CogVideoX), then reconstruct with a two-layer particle representation coupling differentiable Position-Based Fluid simulation with 3D Gaussian splatting visual particles.
Key results
- On FluidNexus-Smoke, novel view synthesis PSNR 30.62 vs 22.23 for the best baseline HyFluid (22.23 is HyFluid's input-view number; PINF 22.68 on ScalarFlow).
- On ScalarFlow, FluidNexus reaches 32.45 PSNR novel view synthesis versus 22.68 for PINF.
- Divergence of velocity field on FluidNexus-Smoke drops to 0.0246 vs 0.0323 for SpacetimeGaussians and 0.0451 for PINF (ScalarFlow: 0.0126 vs 0.0297).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FluidNexus instead of PINF | 3D fluid reconstruction · ScalarFlow | PSNR (novel view synthesis): 22.68 -> 32.45 | Helped |
| use FluidNexus instead of HyFluid | 3D fluid reconstruction · FluidNexus-Smoke | PSNR (novel view synthesis): 22.64 -> 30.62 | Helped |
| use FluidNexus instead of HyFluid | 3D fluid reconstruction · FluidNexus-Ball | PSNR (novel view synthesis): 20.71 -> 29.89 | Helped |
| use FluidNexus instead of HyFluid | 3D fluid reconstruction · ScalarFlow | PSNR (input-view future prediction): 26.84 -> 28.51 | Helped |
| use FluidNexus instead of PINF | 3D fluid reconstruction · FluidNexus-Smoke | PSNR (input-view future prediction): 26.48 -> 27.79 | Helped |
| use FluidNexus instead of SpacetimeGaussians | 3D fluid reconstruction · FluidNexus-Ball | PSNR (input-view future prediction): 25.80 -> 27.70 | Helped |
| remove the novel-view video synthesizer (add it back) (ablation) | FluidNexus · FluidNexus-Smoke | PSNR (novel view synthesis): 21.88 -> 30.62 | Helped |
| remove generative video refinement (add it back) (ablation) | FluidNexus · FluidNexus-Smoke | PSNR (novel view synthesis): 29.12 -> 30.62 | Helped |
| remove long video generation support (add it back) (ablation) | FluidNexus · FluidNexus-Smoke | PSNR (novel view synthesis): 30.14 -> 30.62 | Helped |
| remove the physics loss (add it back) (ablation) | FluidNexus · FluidNexus-Smoke | PSNR (re-simulation): 28.00 -> 30.61 | Helped |
| remove the incompressibility loss (add it back) (ablation) | FluidNexus · FluidNexus-Smoke | PSNR (re-simulation): 28.76 -> 30.61 | Helped |
| remove simulation supervision (add it back) (ablation) | FluidNexus · FluidNexus-Smoke | PSNR (re-simulation): 29.70 -> 30.61 | Helped |
Surprising findings
- Pure physics simulation of reconstructed fluids fails to capture dynamics for future prediction, so generative video refinement is needed in prediction.
Limitations
- Reconstructions on challenging datasets are blurry and miss fine fluid details.
- Does not generalize to different backgrounds or objects because the novel-view synthesizer is trained on limited scenes.
Experimental setup
Datasets: ScalarFlow FluidNexus-Smoke FluidNexus-Ball
Benchmarks: ScalarFlow FluidNexus-Smoke FluidNexus-Ball
Metrics: PSNR SSIM LPIPS divergence of velocity field
Baselines: PINF HyFluid SpacetimeGaussians
Builds on: Zero123 CogVideoX SDEdit Position-Based Fluid 3D Gaussian Splatting
Closest papers by meaning
- OmniPhysGS: 3D Constitutive Gaussians for General Physics-Based Dynamics Generation (ICLR 2025 · similarity 0.85)
- Latent Intuitive Physics: Learning to Transfer Hidden Physics from A 3D Video (ICLR 2024 · similarity 0.84)
- Gaussian Splashing: Unified Particles for Versatile Motion Synthesis and Rendering (CVPR 2025 · similarity 0.84)
- DreamPhysics: Learning Physics-Based 3D Dynamics with Video Diffusion Priors (AAAI 2025 · similarity 0.83)
- FreeGave: 3D Physics Learning from Dynamic Videos by Gaussian Velocity (CVPR 2025 · similarity 0.83)
- Wonderland: Navigating 3D Scenes from a Single Image (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.