Research map / 3D vision / 3D generation
SurMo: Surface-based 4D Motion Modeling for Dynamic Human Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SurMo, a surface-based 4D motion triplane with physical motion (normal/velocity) prediction and a volumetric surface-conditioned renderer for novel-view rendering of dynamic humans from sparse multi-view videos.
Problem
Dynamic human rendering from sparse multi-view video typically maps static poses to images frame by frame, ignoring temporal motion dynamics and so failing to render time-varying secondary clothing motion and shadows.
Approach
Jointly model temporal dynamics and appearance by encoding pose, velocity and motion trajectory on the SMPL UV surface into a surface-based triplane (UV plus signed-distance height), train a physical motion decoder to predict next-step surface normals and velocities, and render with a volumetric surface-conditioned NeRF plus geometry-aware super-resolution.
Key results
- On ZJU-MoCap (6 sequences), SurMo reaches LPIPS 0.075, FID 67.725, SSIM 0.833, PSNR 24.815, beating HumanNeRF (LPIPS 0.106, FID 88.382, SSIM 0.792, PSNR 23.624).
- Ablation on S313: adding dynamics conditioning, velocity prediction and normal prediction improves PSNR from 22.781 to 26.654 and LPIPS from 0.120 to 0.060.
- Surface-based triplane converges faster and handles self-occlusion better than volumetric triplane (EG3D-style) on ZJU-MoCap subjects.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dynamics conditioning (ablation) | SurMo with pose conditioning · ZJU-MoCap (S313) | LPIPS: .120 -> .085 | Helped |
| add dynamics conditioning (ablation) | SurMo with pose conditioning · ZJU-MoCap (S313) | FID: 104.83 -> 73.674 | Helped |
| add dynamics conditioning (ablation) | SurMo with pose conditioning · ZJU-MoCap (S313) | SSIM: .793 -> .834 | Helped |
| add dynamics conditioning (ablation) | SurMo with pose conditioning · ZJU-MoCap (S313) | PSNR: 22.781 -> 24.908 | Helped |
| add surface velocity prediction (ablation) | SurMo with pose and dynamics conditioning · ZJU-MoCap (S313) | LPIPS: .085 -> .069 | Helped |
| add surface velocity prediction (ablation) | SurMo with pose and dynamics conditioning · ZJU-MoCap (S313) | FID: 73.674 -> 62.092 | Helped |
| add surface velocity prediction (ablation) | SurMo with pose and dynamics conditioning · ZJU-MoCap (S313) | SSIM: .834 -> .856 | Helped |
| add surface velocity prediction (ablation) | SurMo with pose and dynamics conditioning · ZJU-MoCap (S313) | PSNR: 24.908 -> 25.845 | Helped |
| add surface normal prediction (ablation) | SurMo with pose and dynamics conditioning and surface velocity prediction · ZJU-MoCap (S313) | LPIPS: .069 -> .060 | Helped |
| add surface normal prediction (ablation) | SurMo with pose and dynamics conditioning and surface velocity prediction · ZJU-MoCap (S313) | FID: 62.092 -> 50.170 | Helped |
| add surface normal prediction (ablation) | SurMo with pose and dynamics conditioning and surface velocity prediction · ZJU-MoCap (S313) | SSIM: .856 -> .869 | Helped |
| add surface normal prediction (ablation) | SurMo with pose and dynamics conditioning and surface velocity prediction · ZJU-MoCap (S313) | PSNR: 25.845 -> 26.654 | Helped |
Surprising findings
- Several baseline methods mainly generate pose-dependent rather than time-varying appearances, e.g., HumanNeRF renders nearly identical wrinkles for similar poses.
Limitations
- Relies on estimated SMPL body meshes and pose estimation, which introduce errors affecting consecutive-frame velocity.
- Less sensitive to small velocities because consecutive frames may share identical pose fits.
Experimental setup
Datasets: ZJU-MoCap MPII-RDDC AIST++
Benchmarks: ZJU-MoCap MPII-RDDC AIST++
Metrics: SSIM PSNR LPIPS FID
Baselines: Neural Body HumanNeRF Instant-NVR ARAH DVA HVTR++
Builds on: EG3D Neural Body HumanNeRF SMPL GIRAFFE
Closest papers by meaning
- Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans From a Single Camera (CVPR 2022 · similarity 0.91)
- 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.88)
- Real-time Photorealistic Dynamic Scene Representation and Rendering with 4D Gaussian Splatting (ICLR 2024 · similarity 0.87)
- Natural Human Motion Recovery by Aligning High-Order Temporal Dynamics from Monocular Videos (CVPR 2026 · similarity 0.87)
- Scaling Large Motion Models with Million-Level Human Motions (ICML 2025 · similarity 0.87)
- PKU-DyMVHumans: A Multi-View Video Benchmark for High-Fidelity Dynamic Human Modeling (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.