Research map / 3D vision / Face and shape
HumanNeRF-SE: A Simple yet Effective Approach to Animate HumanNeRF with Diverse Poses
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HumanNeRF-SE, a monocular/few-shot animatable human NeRF that combines SMPL-based sparse-conv voxel filtering, frozen SMPL blend weights for rigid deformation, and point-level features for non-rigid refinement, cutting learnable parameters and training time.
Problem
Animating human NeRFs to synthesize novel poses that differ strongly from training videos, from monocular or few-shot input, where learnable blend weights overfit and training is expensive.
Approach
Combine explicit SMPL vertices with implicit NeRF: a sparse-convolution voxel volume of SMPL vertices filters irrelevant sampling points and yields point-level spatial features; rigid deformation uses frozen pre-trained SMPL blend weights (nearest vertex) and a point-level refinement network replaces frame-level conditioning.
Key results
- On ZJU-MOCAP few-shot, HumanNeRF-SE reaches PSNR 30.11 / SSIM 0.9684 / LPIPS*1000 32.084 vs HumanNeRF 29.90 / 0.9683 / 33.056 and MonoHuman 30.10 / 0.9677 / 36.494.
- Training takes 2.5 h versus 56 h for HumanNeRF and 70 h for MonoHuman; rendering 0.16 s/it versus 2.37 s/it for HumanNeRF.
- Uses less than 1% learnable parameters and reports about 15x faster synthesis without acceleration modules; subjective study with six or more participants rated its results perfect in significantly more cases than HumanNeRF.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HumanNeRF-SE instead of HumanNeRF | HumanNeRF-SE · ZJU-MOCAP full | LPIPS (×1000): 24.822 -> 24.085 | Helped |
| Use HumanNeRF-SE instead of MonoHuman | HumanNeRF-SE · ZJU-MOCAP full | LPIPS (×1000): 31.292 -> 24.085 | Helped |
| Use HumanNeRF-SE instead of HumanNeRF | HumanNeRF-SE · ZJU-MOCAP few-shot | LPIPS (×1000): 33.056 -> 32.084 | Helped |
| Use HumanNeRF-SE instead of MonoHuman | HumanNeRF-SE · ZJU-MOCAP few-shot | LPIPS (×1000): 36.494 -> 32.084 | Helped |
| Use HumanNeRF-SE instead of HumanNeRF | HumanNeRF-SE · In-the-wild full | PSNR: 28.97 -> 29.23 | Helped |
| Use HumanNeRF-SE instead of MonoHuman | HumanNeRF-SE · In-the-wild full | PSNR: 29.15 -> 29.23 | Helped |
| Use HumanNeRF-SE instead of HumanNeRF | HumanNeRF-SE · In-the-wild few-shot | LPIPS (×1000): 50.240 -> 47.161 | Helped |
| Use HumanNeRF-SE instead of MonoHuman | HumanNeRF-SE · In-the-wild few-shot | LPIPS (×1000): 56.220 -> 47.161 | Helped |
| Add point-feature refinement (ablation) | HumanNeRF-SE · ZJU-MOCAP | PSNR: 30.93 -> 31.09 | Helped |
| Add point-feature refinement (ablation) | HumanNeRF-SE · ZJU-MOCAP | LPIPS (×1000): 24.712 -> 24.085 | Helped |
| Add Conv-Filter (ablation) | HumanNeRF-SE · ZJU-MOCAP | PSNR: 9.22 -> 31.09 | Helped |
| Add Conv-Filter (ablation) | HumanNeRF-SE · ZJU-MOCAP | LPIPS (×1000): 318.460 -> 24.085 | Helped |
Surprising findings
- Learnable blend weights tend to assign negative values to irrelevant joints, and mapping them via sigmoid reproduces the same artifacts, suggesting the issue is distribution rather than filtering alone.
- Without Conv-Filter point filtering, PSNR collapses to 9.22 on ZJU-MOCAP ablation, showing filtering is essential.
Limitations
- Performance depends on accuracy of estimated SMPL; blurring when SMPL is inaccurate.
- Voxelization can cause image edge serrations; fixing requires larger voxel/kernel cost.
- Relies on basic SMPL, so hand and facial details are hard to drive.
Experimental setup
Datasets: ZJU-MOCAP in-the-wild handheld videos (self-captured)
Benchmarks: ZJU-MOCAP in-the-wild data
Metrics: PSNR SSIM LPIPS training time rendering time subjective score
Baselines: HumanNeRF MonoHuman Ani-NeRF
Builds on: HumanNeRF NeuralBody Ani-NeRF MonoHuman SMPL
Closest papers by meaning
- HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation (NeurIPS 2024 · similarity 0.86)
- FlexPose: Pose Distribution Adaptation with Limited Guidance (AAAI 2025 · similarity 0.85)
- HumanDreamer: Generating Controllable Human-Motion Videos via Decoupled Generation (CVPR 2025 · similarity 0.85)
- REACTO: Reconstructing Articulated Objects from a Single Video (CVPR 2024 · similarity 0.85)
- 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting (CVPR 2024 · similarity 0.85)
- DisPose: Disentangling Pose Guidance for Controllable Human Image Animation (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.