Research map / 3D vision / Gaussian splatting
Learnable Infinite Taylor Gaussian for Dynamic View Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a learnable infinite Taylor formulation for 4D Gaussian splatting, combining third-order polynomial transformation fields with a GP/LP Peano-remainder network via LBS, for dynamic novel view synthesis on N3DV and Technicolor.
Problem
Modeling the time-varying position, rotation, and scale of Gaussians in dynamic scenes from limited multi-view photometric data, where end-to-end networks lack explicit supervision and polynomial models need handcrafted design and generalize poorly.
Approach
Decomposes each Gaussian's motion into a dominant third-order Taylor polynomial (position, scale, quaternion orientation over time) plus a learned Peano remainder predicted by an MLP on sparse farthest-point Global Gaussian primitives, propagated to Local primitives via linear blend skinning with learnable radii.
Key results
- On N3DV, the method reports PSNR 32.59 on Cook Spinach, 33.12 on Sear Steak, 33.34 on Flame Steak, and 33.06 on Cut Roast Beef, the best among compared methods.
- On Technicolor Birthday, PSNR 34.72 vs. 33.87 for STG and 33.81 for D3DGS.
- Ablation on N3DV: removing time-motion drops PSNR from 33.03 to 29.24; removing the Peano remainder yields 31.51.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the learnable infinite Taylor Gaussian method instead of SWinGS | Dynamic view rendering · N3DV — Cook Spinach | PSNR: 31.96 -> 32.59 | Helped |
| use the learnable infinite Taylor Gaussian method instead of SWinGS | Dynamic view rendering · N3DV — Sear Steak | PSNR: 32.21 -> 33.12 | Helped |
| use the learnable infinite Taylor Gaussian method instead of NeRFPlayer | Dynamic view rendering · N3DV — Flame Steak | PSNR: 31.93 -> 33.34 | Helped |
| use the learnable infinite Taylor Gaussian method instead of HyperReel | Dynamic view rendering · N3DV — Cut Roast Beef | PSNR: 32.25 -> 33.06 | Helped |
| use the learnable infinite Taylor Gaussian method instead of STG | Dynamic view rendering · Technicolor — Birthday | PSNR: 33.87 -> 34.72 | Helped |
| use the learnable infinite Taylor Gaussian method instead of D3DGS | Dynamic view rendering · Technicolor — Painter | PSNR: 37.38 -> 38.37 | Helped |
| use the learnable infinite Taylor Gaussian method instead of STG | Dynamic view rendering · Technicolor — Train | PSNR: 33.36 -> 35.30 | Helped |
| use the learnable infinite Taylor Gaussian method instead of D3DGS | Dynamic view rendering · Technicolor — Fatma | PSNR: 38.40 -> 38.91 | Helped |
| add time-dependent opacity (ablation) | Learnable infinite Taylor Gaussian · N3DV | PSNR: 31.17 -> 33.03 | Helped |
| add time-dependent motion modeling (ablation) | Learnable infinite Taylor Gaussian · N3DV | PSNR: 29.24 -> 33.03 | Helped |
| add time-dependent rotation modeling (ablation) | Learnable infinite Taylor Gaussian · N3DV | PSNR: 31.21 -> 33.03 | Helped |
| add time-dependent scale modeling (ablation) | Learnable infinite Taylor Gaussian · N3DV | PSNR: 31.40 -> 33.03 | Helped |
Surprising findings
- Table 2 reports D3DGS achieving the best PSNR on Painter and Fatma, which the text acknowledges, so the method is not best on every sequence.
Limitations
- Current model may need refinement for more complex and highly dynamic scenes.
- Performance depends on COLMAP-derived initial point clouds and camera poses.
Experimental setup
Datasets: Neural 3D Video (N3DV) Technicolor Light Field Dataset
Benchmarks: N3DV Technicolor Light Field Dataset
Metrics: PSNR SSIM LPIPS
Baselines: MixVoxels K-Planes HexPlane HyperReel NeRFPlayer StreamRF SWinGS D3DGS 4DGS SCGS STG FSGS
Builds on: 3D Gaussian Splatting 4DGS D3DGS HexPlane
Closest papers by meaning
- Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting (ICLR 2024 · similarity 0.86)
- Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction (CVPR 2024 · similarity 0.85)
- 4D Gaussian Splatting for Real-Time Dynamic Scene Rendering (CVPR 2024 · similarity 0.85)
- Spacetime Gaussian Feature Splatting for Real-Time Dynamic View Synthesis (CVPR 2024 · similarity 0.85)
- Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle (CVPR 2024 · similarity 0.85)
- Fully Explicit Dynamic Gaussian Splatting (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.