Research map / Image generation / Text-to-image diffusion
Hand1000: Generating Realistic Hands from Text with Only 1,000 Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Hand1000 fine-tunes Stable Diffusion v1.4 on 1,000 gesture images per target gesture, using MediaPipe-derived mean gesture features fused with CLIP text embeddings, and releases a text-hand dataset built from HaGRID with BLIP2 captions refined by LLaMA3.
Problem
Text-to-image diffusion models generate anatomically incorrect hands (wrong finger counts, twisted or blurred fingers), and existing fixes need hundreds of thousands of training images.
Approach
Inject a hand-gesture feature from a pretrained MediaPipe hand recognizer into the CLIP text embedding (via a frozen FC fusion and linear mixing), optimize that embedding with a frozen SD model, then fine-tune SD conditioned on the frozen optimized embedding, borrowing the embedding-then-model-tuning recipe from Imagic-style image editing.
Key results
- Hand1000 reaches FID-H 78.960 and HAND-CONF 0.948, versus 109.168 and 0.895 for Stable Diffusion, and 123.254 and 0.913 for fine-tuned Stable Diffusion.
- Compared with HandRefiner, Hand1000 lowers FID from 97.565 to 86.169 and FID-H from 88.997 to 78.960.
- Fine-tuned Stable Diffusion achieves the best FID (76.120) and KID (0.040), which the authors note are less reliable for hands.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Hand1000 instead of Stable Diffusion | SDv1.4 · HaGRID hand image generation | FID-H: 109.168 -> 78.960; 30.208 reduction | Helped |
| Use Hand1000 instead of fine-tuned Stable Diffusion | SDv1.4 · HaGRID hand image generation | FID: 76.120 -> 86.169 | Hurt |
| Use Hand1000 instead of fine-tuned Stable Diffusion | SDv1.4 · HaGRID hand image generation | FID-H: 123.254 -> 78.960; 44.294 reduction | Helped |
| Use Hand1000 instead of HandRefiner | SDv1.4 · HaGRID hand image generation | FID-H: 88.997 -> 78.960; 10.037 reduction | Helped |
| Post-process captions with LLaMA3 | BLIP2 captions · HaGRID captions | Image-Caption Consistency: 0.288 -> 0.298 | Helped |
| Add embedding optimization (ablation) | Hand1000 · HaGRID hand image generation ablation | FID-H: 108.596 -> 83.593; 25.003 reduction | Helped |
| Add embedding optimization (ablation) | Hand1000 · HaGRID hand image generation ablation | KID-H: 0.104 -> 0.069; 0.035 reduction | Helped |
| Add feature fusion (ablation) | Hand1000 · HaGRID hand image generation ablation | FID-H: 108.596 -> 88.277; 20.319 reduction | Helped |
| Add feature fusion (ablation) | Hand1000 · HaGRID hand image generation ablation | KID-H: 0.104 -> 0.056; 0.048 reduction | Helped |
| Use embedding optimization and feature fusion together (ablation) | Hand1000 · HaGRID hand image generation ablation | FID-H: 108.596 -> 78.960; 29.636 reduction | Helped |
| Use embedding optimization and feature fusion together (ablation) | Hand1000 · HaGRID hand image generation ablation | KID-H: 0.104 -> 0.053; 0.051 reduction | Helped |
| Set λ too low (ablation) | Hand1000 · Phone call gesture | Hand image quality | Mixed |
Surprising findings
- Fine-tuned Stable Diffusion gets better FID and KID than Hand1000 yet much worse hand-specific metrics, so the authors argue FID/KID are unreliable for hand quality.
Limitations
- Does not address simultaneous multi-gesture generation or hand-object interaction, due to lack of datasets.
- Possible overfitting of the embedding at training set size 1,200 and above, per the authors.
- Gesture-specific hyperparameter lambda must be tuned per gesture and dataset.
Experimental setup
Datasets: HaGRID
Benchmarks: HaGRID-derived hand gesture test split
Metrics: FID KID FID-H KID-H HAND-CONF
Baselines: Stable Diffusion Stable Diffusion (fine-tuned) HandRefiner Imagic
Builds on: Imagic Stable Diffusion HandRefiner HanDiffuser DDIM
Closest papers by meaning
- HanDiffuser: Text-to-Image Generation With Realistic Hand Appearances (CVPR 2024 · similarity 0.91)
- FoundHand: Large-Scale Domain-Specific Learning for Controllable Hand Image Generation (CVPR 2025 · similarity 0.88)
- SiMHand: Mining Similar Hands for Large-Scale 3D Hand Pose Pre-training (ICLR 2025 · similarity 0.87)
- BOTH2Hands: Inferring 3D Hands from Both Text Prompts and Body Dynamics (CVPR 2024 · similarity 0.87)
- HandVQA: Diagnosing and Improving Fine-Grained Spatial Reasoning about Hands in Vision-Language Models (CVPR 2026 · similarity 0.86)
- Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interaction (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.