Research map / 3D vision / Gaussian splatting
Gaussian Splatting Lucas-Kanade
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adapts the classical Lucas-Kanade method to dynamic Gaussian splatting, deriving an analytical velocity field from a forward warp field and time-integrating it to impose scene-flow (optical flow and depth) constraints for reconstructing dynamic scenes from near-static monocular cameras.
Problem
Dynamic Gaussian splatting reconstructs poorly from monocular video with minimal camera motion and highly dynamic objects, where data-driven depth and optical flow priors produce biased geometry and motion.
Approach
Derive an analytical velocity field directly from a generic forward warp field via a modified Lucas-Kanade expression (Taylor expansion with pseudoinverse Jacobian), time-integrate it with an ODE solver to get Gaussian offsets, and supervise rendered optical flow and depth against RAFT flow and Marigold-based monocular depth ranking, without extra learnable parameters.
Key results
- On the Dynamic Scenes Playground scene, the method reaches 26.34 PSNR versus 24.82 for Deformable GS.
- On HyperNeRF, the method reports 27.38 PSNR and 0.26 LPIPS, versus 26.98 PSNR for 4DGaussians.
- On Plenoptic Videos, the method reports a mean PSNR of 33.44 versus 32.46 for Deformable GS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add analytical scene-flow regularization | Dynamic Gaussian Splatting · Dynamic Scenes: Playground | PSNR: 24.82 -> 26.34 | Helped |
| add analytical scene-flow regularization | Dynamic Gaussian Splatting · Hypernerf | PSNR: 26.98 -> 27.38 | Helped |
| add analytical scene-flow regularization | Dynamic Gaussian Splatting · Plenoptic Videos: Mean | PSNR: 32.46 -> 33.44 | Helped |
| add analytical scene-flow regularization | Dynamic Gaussian Splatting · DyCheck | PSNR: 17.54 -> 16.03; 18.38 -> 16.71; 16.61 -> 15.46; 13.65 -> 13.60; 18.38 -> 17.41 | Hurt |
| add depth supervision alongside flow supervision (ablation) | Ours · Dynamic Scenes: Playground | PSNR: 24.37 -> 26.34 | Helped |
| add flow supervision alongside depth supervision (ablation) | Ours · Dynamic Scenes: Playground | PSNR: 24.76 -> 26.34 | Helped |
| apply direct flow supervision (ablation) | Deformable GS · Dynamic Scenes: Playground | PSNR: 24.82 -> 20.97 | Hurt |
| apply direct depth supervision (ablation) | Deformable GS · Dynamic Scenes: Playground | PSNR: 24.82 -> 21.17 | Hurt |
| apply direct flow and depth supervision (ablation) | Deformable GS · Dynamic Scenes: Playground | PSNR: 24.82 -> 23.94 | Hurt |
| use refined camera poses (ablation) | Ours · DyCheck | PSNR: 16.03 -> 22.61; 16.71 -> 24.09; 15.46 -> 23.27; 13.60 -> 17.78; 17.41 -> 19.91 | Helped |
| replace Marigold depth with Depth Anything V2 (ablation) | Ours · Dynamic Scenes: Playground | PSNR: 26.34 -> 26.96 | Helped |
| replace Marigold depth with Midas (ablation) | Ours · Dynamic Scenes: Playground | PSNR: 26.34 -> 26.78 | Helped |
Surprising findings
- Directly adding depth or flow supervision to Deformable GS degrades results on Playground (e.g., 24.76 and 24.37 PSNR vs 24.82 baseline), suggesting overfitting to training viewpoints.
Limitations
- Sensitive to SfM initialization, failing when dynamic regions lack initialized points.
- Sensitive to inaccurate bundle-adjusted camera parameters.
- Pseudo-inverse computation adds computational complexity.
Experimental setup
Datasets: Dynamic Scenes Plenoptic Video HyperNeRF DyCheck DAVIS 2017
Benchmarks: Dynamic Scenes Plenoptic Video HyperNeRF DyCheck
Metrics: PSNR SSIM LPIPS
Baselines: Deformable GS 4DGaussians 3DGS MA-GS NSFF NR-NeRF Nerfies Flow-supervised NeRF Marbles
Builds on: 3D Gaussian Splatting Deformable GS Lucas-Kanade method RAFT Marigold
Closest papers by meaning
- MotionGS: Exploring Explicit Motion Guidance for Deformable 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.92)
- Fully Explicit Dynamic Gaussian Splatting (NeurIPS 2024 · similarity 0.92)
- Motion Decoupled 3D Gaussian Splatting for Dynamic Object Representation (AAAI 2025 · similarity 0.91)
- H3D-DGS: Exploring Heterogeneous 3D Motion Representation for Deformable 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.90)
- HAIF-GS: Hierarchical and Induced Flow-Guided Gaussian Splatting for Dynamic Scene (NeurIPS 2025 · similarity 0.90)
- Efficient Gaussian Splatting for Monocular Dynamic Scene Rendering via Sparse Time-Variant Attribute Modeling (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.