Research map / 3D vision / NeRF and radiance fields
3D Visibility-Aware Generalizable Neural Radiance Fields for Interacting Hands
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VA-NeRF, a single-image generalizable NeRF for interacting hands that fuses features via visibility-aware weighting and trains with a pixel-wise visibility-map discriminator.
Problem
Generalizable NeRF synthesis of interacting hands from a single image, where inter-hand occlusions and large view variations make existing human-oriented NeRFs unreliable.
Approach
A visibility-aware feature fusion module adaptively weights pixel-aligned features of the query point, nearest MANO mesh vertex, its mirrored vertex on the other hand, and global hand features; a visibility-guided discriminator predicts pixel-wise visibility maps to supervise invisible regions adversarially.
Key results
- On InterHand2.6M, VA-NeRF reaches 25.01 PSNR, 0.86 SSIM, 0.21 LPIPS, versus 23.68 PSNR / 0.83 SSIM / 0.27 LPIPS for the best baseline HandNeRF.
- Under large view variations (>30 degrees), VA-NeRF achieves 24.23 PSNR versus 22.98 for NHP.
- With 0.3 occlusion mask ratio, VA-NeRF keeps 24.93 PSNR while KeypointNeRF drops to 21.38.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VA-NeRF instead of the strongest reported baseline | Interacting-hand NeRF synthesis · Interhand2.6M | PSNR: 23.68 -> 25.01 | Helped |
| Use VA-NeRF instead of the strongest reported baseline | Interacting-hand NeRF synthesis · Interhand2.6M | LPIPS: 0.27 -> 0.21 | Helped |
| Add nearest mesh vertex feature p to query-point feature q (ablation) | VA-NeRF feature fusion · Interhand2.6M feature ablation | PSNR: 24.86 -> 25.06 | Helped |
| Add mirrored mesh vertex feature p' to query-point feature q (ablation) | VA-NeRF feature fusion · Interhand2.6M feature ablation | PSNR: 24.86 -> 25.47 | Helped |
| Add both nearest and mirrored mesh vertex features p and p' (ablation) | VA-NeRF feature fusion · Interhand2.6M feature ablation | PSNR: 24.86 -> 25.74 | Helped |
| Remove visibility guidance from attention weights (ablation) | VA-NeRF feature fusion · Interhand2.6M attention ablation | PSNR: 25.74 -> 24.74 | Hurt |
| Remove visibility guidance from attention weights (ablation) | VA-NeRF feature fusion · Interhand2.6M attention ablation | LPIPS: 0.18 -> 0.20 | Hurt |
| Replace visibility-guided discriminator with binary discriminator (ablation) | VA-NeRF adversarial learning · View variation | PSNR: 24.34 -> 23.77 | Hurt |
| Replace visibility-guided discriminator with binary discriminator (ablation) | VA-NeRF adversarial learning · View variation | LPIPS: 0.21 -> 0.24 | Hurt |
| Replace visibility-guided discriminator with binary discriminator (ablation) | VA-NeRF adversarial learning · Occlusion | PSNR: 25.43 -> 24.57 | Hurt |
| Replace visibility-guided discriminator with binary discriminator (ablation) | VA-NeRF adversarial learning · Occlusion | LPIPS: 0.19 -> 0.20 | Hurt |
Limitations
- Relies on MANO hand mesh fitting from the input image, so errors in mesh estimation propagate to features.
- Assumes both hands share the same MANO vertex topology for mirrored-vertex lookup.
Experimental setup
Datasets: InterHand2.6M
Benchmarks: InterHand2.6M
Metrics: PSNR SSIM LPIPS
Baselines: SMPLpix KeypointNeRF NHP HandNeRF
Builds on: NeRF KeypointNeRF NHP MANO HandNeRF
Closest papers by meaning
- Learning Interaction-aware 3D Gaussian Splatting for One-shot Hand Avatars (NeurIPS 2024 · similarity 0.87)
- Single-view Image to Novel-view Generation for Hand-Object Interactions (AAAI 2025 · similarity 0.86)
- Fine-Grained Multi-View Hand Reconstruction Using Inverse Rendering (AAAI 2024 · similarity 0.86)
- SRHand: Super-Resolving Hand Images and 3D Shapes via View/Pose-aware Neural Image Representations and Explicit Meshes (NeurIPS 2025 · similarity 0.86)
- URHand: Universal Relightable Hands (CVPR 2024 · similarity 0.86)
- Learning Robust Generalizable Radiance Field with Visibility and Feature Augmented Point Representation (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.