Research map / 3D vision / Face and shape
SRHand: Super-Resolving Hand Images and 3D Shapes via View/Pose-aware Neural Image Representations and Explicit Meshes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
SRHand jointly super-resolves low-resolution multi-view hand images with a normal-conditioned implicit image function (GIIF) and reconstructs high-resolution explicit hand meshes, fine-tuning the SR module for view/pose consistency.
Problem
Reconstructing detailed animatable 3D hand avatars from low-resolution multi-view/multi-pose images, where existing multi-view SR methods target static scenes and fixed scales and hand mesh methods need high-resolution inputs.
Approach
A geometric-aware implicit image function (LIIF extended with MANO-derived normal maps via an hourglass encoder and GAN loss) is adaptively fine-tuned using consistency, wavelet frequency, and discriminator losses computed from an explicit subdivided-mesh (XHand-style) reconstruction, so image SR and 3D shape mutually enforce multi-view and pose consistency.
Key results
- On InterHand2.6M, GIIF with fine-tuning reaches 30.06 PSNR (SR) and 2.16 mm P2P, versus 27.47 PSNR for LIIF and 3.43 mm for GIIF-XHand without fine-tuning.
- At x16 upscaling on InterHand2.6M, GIIF achieves 31.60 PSNR / 0.0637 LPIPS, beating LIIF* (29.85 / 0.0996) and LIIF (25.76 / 0.1848).
- Against multi-view SR NeRF methods at x4 on a static-scene setting, SRHand gets the best LPIPS (0.0475) though SRGS has higher PSNR (32.74 vs 29.06).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune GIIF jointly with the reconstructed 3D shape | SRHand · InterHand2.6M | PSNR: 29.17 -> 29.88 | Helped |
| Fine-tune GIIF jointly with the reconstructed 3D shape | SRHand · Goliath | PSNR: 23.50 -> 24.31 | Helped |
| Add MANO normal-map conditioning | LIIF · InterHand2.6M SR ×16 | PSNR: 25.76 -> 29.85 | Helped |
| Add MANO normal-map conditioning | IDM · InterHand2.6M SR ×16 | LPIPS: 0.3603 -> 0.0970 | Helped |
| Replace XHand with the enhanced mesh representation (ablation) | 3D hand reconstruction · InterHand2.6M reconstruction, Identity 1 | PSNR: 27.77 -> 28.92 | Helped |
| Replace XHand with the enhanced mesh representation (ablation) | 3D hand reconstruction · InterHand2.6M reconstruction, Identity 2 | LPIPS: 0.0338 -> 0.0303 | Helped |
| Apply consistency loss alone during adaptive fine-tuning (ablation) | SRHand · InterHand2.6M fine-tuning ablation | PSNR (SR): 29.96 -> 23.94 | Hurt |
| Add wavelet frequency-maintenance loss to consistency loss (ablation) | SRHand · InterHand2.6M fine-tuning ablation | PSNR (SR): 23.94 -> 29.93 | Helped |
| Add wavelet frequency-maintenance loss to consistency loss (ablation) | SRHand · InterHand2.6M fine-tuning ablation | P2P (mm): 3.72 -> 2.41 | Helped |
| Add discriminator loss to consistency and wavelet losses (ablation) | SRHand · InterHand2.6M fine-tuning ablation | PSNR: 28.97 -> 29.88 | Helped |
| Add discriminator loss to consistency and wavelet losses (ablation) | SRHand · InterHand2.6M fine-tuning ablation | LPIPS: 0.0410 -> 0.0362 | Helped |
| Use noisy rather than ground-truth MANO parameters (ablation) | SRHand · InterHand2.6M super-resolution | PSNR: 30.06 -> 30.11 | No clear effect |
Surprising findings
- Adding the consistency loss alone degrades PSNR sharply (29.96 to 23.94 SR PSNR), pushing textures toward a mean value; the wavelet and discriminator losses are needed to recover detail.
Limitations
- Relies on ground-truth MANO template parameters, which the paper acknowledges is a common but strong assumption.
- Goliath has only 4 subjects, so the SR module trained on InterHand2.6M is reused there; the multi-view SR comparisons are done on static single-pose settings.
- SRGS outperforms SRHand in PSNR at x4 on the static setting, and the 3D consistency comparisons use only a single pose.
Experimental setup
Datasets: InterHand2.6M Goliath
Benchmarks: InterHand2.6M Goliath
Metrics: PSNR SSIM LPIPS P2P (mm) inconsistency MPJPE
Baselines: Bicubic LIIF IDM Real-ESRGAN SwinIR UHM XHand NeRF-SR DiSR-NeRF SRGS
Builds on: LIIF LIIF-GAN XHand MANO SuperNeRF
Closest papers by meaning
- Fine-Grained Multi-View Hand Reconstruction Using Inverse Rendering (AAAI 2024 · similarity 0.90)
- SiMA-Hand: Boosting 3D Hand-Mesh Reconstruction by Single-to-Multi-View Adaptation (AAAI 2024 · similarity 0.89)
- HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Video (CVPR 2024 · similarity 0.89)
- Learning Explicit Contact for Implicit Reconstruction of Hand-Held Objects from Monocular Images (AAAI 2024 · similarity 0.89)
- HandBooster: Boosting 3D Hand-Mesh Reconstruction by Conditional Synthesis and Sampling of Hand-Object Interactions (CVPR 2024 · similarity 0.89)
- FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image Generation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.