Research map / 3D vision / Gaussian splatting
Learning Interaction-aware 3D Gaussian Splatting for One-shot Hand Avatars
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a two-stage interaction-aware 3D Gaussian Splatting framework that creates animatable two-hand avatars from a single image on InterHand2.6M, combining learned cross-subject hand priors with per-subject optimized identity maps.
Problem
Creating animatable interacting-hand avatars from a single image is hard due to limited views, varied poses, self- and inter-hand occlusion, and out-of-distribution hand identities.
Approach
Disentangles hand representation into learned geometric features and neural texture maps (trained across subjects) plus optimization-based identity maps fitted one-shot per subject; adds an interaction-aware attention module that applies self-attention to points flagged by mesh-neighborhood differences from a canonical mesh, and a self-adaptive Gaussian refinement module that prunes/splits Gaussia
Key results
- On InterHand2.6M one-shot novel view synthesis, the method reaches 26.14 PSNR / 0.869 SSIM / 0.161 LPIPS, versus 25.38 PSNR for VANeRF and 25.31 for OHTA*.
- On novel pose synthesis it reaches 26.56 PSNR / 0.890 SSIM / 0.133 LPIPS, versus 25.93 PSNR for OHTA*.
- Ablations show the identity map improves stage-one PSNR from 27.76 (w/o IMap) to 28.11, and the full model's one-shot PSNR is 26.56 vs 25.93 for the OHTA* baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the full interaction-aware Gaussian splatting method instead of VANeRF | One-shot hand avatar reconstruction · InterHand2.6M novel view synthesis | PSNR: 25.38 -> 26.14 | Helped |
| Use the full interaction-aware Gaussian splatting method instead of OHTA* | One-shot hand avatar reconstruction · InterHand2.6M novel pose synthesis | LPIPS: 0.156 -> 0.133 | Helped |
| Add an identity map instead of using an identity code (ablation) | IGSN · InterHand2.6M train/capture0 evaluation | PSNR: 27.76 -> 28.11 | Helped |
| Add interaction-aware attention (ablation) | IGSN · InterHand2.6M train/capture0 evaluation | PSNR: 27.52 -> 28.11 | Helped |
| Use refined Gaussian points rather than upsampled hand-mesh points (ablation) | IGSN · InterHand2.6M train/capture0 evaluation | LPIPS: 0.177 -> 0.130 | Helped |
| Add identity inversion (ablation) | One-shot fitting · InterHand2.6M novel pose synthesis | PSNR: 26.32 -> 26.56 | Helped |
| Add color calibration (ablation) | One-shot fitting · InterHand2.6M novel pose synthesis | PSNR: 26.50 -> 26.56 | Helped |
| Add neural texture map bias (ablation) | One-shot fitting · InterHand2.6M novel pose synthesis | PSNR: 26.50 -> 26.56 | Helped |
| Use an identity map instead of an identity code (ablation) | One-shot fitting · InterHand2.6M novel pose synthesis | PSNR: 26.14 -> 26.56 | Helped |
| Use images of both hands rather than a single hand (ablation) | One-shot reconstruction · InterHand2.6M novel view synthesis | PSNR: 24.81 -> 26.14 | Helped |
| Use SAM masks rather than mesh masks (ablation) | One-shot reconstruction · InterHand2.6M novel view synthesis | PSNR: 25.99 -> 26.52 | Helped |
| Use ground-truth rather than ACR-estimated hand mesh parameters (ablation) | One-shot reconstruction · InterHand2.6M novel view synthesis | PSNR: 25.83 -> 26.14 | Helped |
Limitations
- Per-identity optimization still required, so not a single forward-pass method.
- Performance degrades with serious pose estimation errors causing misaligned appearance.
- Separate per-identity optimization prevents sharing information across similar identities.
Experimental setup
Datasets: InterHand2.6M
Benchmarks: InterHand2.6M
Metrics: PSNR SSIM LPIPS
Baselines: KeypointNeRF SMPLpix VANeRF OHTA
Builds on: 3D Gaussian Splatting MANO OHTA VANeRF
Closest papers by meaning
- Single-view Image to Novel-view Generation for Hand-Object Interactions (AAAI 2025 · similarity 0.91)
- BIGS: Bimanual Category-agnostic Interaction Reconstruction from Monocular Videos via 3D Gaussian Splatting (CVPR 2025 · similarity 0.90)
- HOGSA: Bimanual Hand-Object Interaction Understanding with 3D Gaussian Splatting Based Data Augmentation (AAAI 2025 · similarity 0.90)
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.89)
- HUGS: Human Gaussian Splats (CVPR 2024 · similarity 0.89)
- OHTA: One-shot Hand Avatar via Data-driven Implicit Priors (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.