Research map / 3D vision / 3D generation
CLUTCH: Contextualized Language model for Unlocking Text-Conditioned Hand motion modelling in the wild
Balamurugan Thambiraja, Omid Taheri, Radek Danecek, Giorgio Becherini, Gerard Pons-Moll, Justus Thies
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add geometric refinement (Gumbel-Softmax reconstruction loss) stage (ablation) | CLUTCH · 3D-HIW test, text-to-motion | KID: 0.297 -> 0.216 | Improved |
| add geometric refinement stage (ablation) | CLUTCH · 3D-HIW test, text-to-motion | RP3: 0.690 -> 0.721 | Improved |
| add instruction fine-tuning after pre-training (ablation) | CLUTCH · 3D-HIW test, text-to-motion | RP3: 0.53 -> 0.69 | Improved |
| use Gumbel-Softmax geometric refinement instead of EgoLM soft-blending regression (ablation) | CLUTCH · 3D-HIW test, text-to-motion | KID: 0.263 -> 0.216 | Improved |
| geometric refinement on motion-to-text captioning (ablation) | CLUTCH · 3D-HIW test, motion-to-text | RP3: 0.568 -> 0.571 | No clear effect |
| use SHIFT decomposed tokenizer (trajectory/pose and left/right) instead of standard VQ-VAE (ablation) | CLUTCH tokenizer · 3D-HIW motion reconstruction | MPJPE: 93.258 -> 45.944 | Improved |
| left/right hand separation only (PD VQ-VAE) without trajectory/pose split (ablation) | VQ-VAE tokenizer · 3D-HIW motion reconstruction | MPJPE: 93.258 -> 95.266 | No clear effect |
| enlarge VQ-VAE codebook from 4K to 8K for standard VQ-VAE (ablation) | Std VQ-VAE · 3D-HIW motion reconstruction | MPJPE: 93.258 -> 92.150 | No clear effect |
| two-stage parallel chain-of-thought annotation instead of single naive prompt (ablation) | VILA annotation · 3D-HIW annotation quality | GPT-Score: 5.5 -> 6.9 | Improved |
| add closed-vocabulary stage 2 after stage 1 (ablation) | VILA annotation · 3D-HIW annotation quality | GPT-Score: 6.4 -> 6.9 | Improved |
| use combined high-level and fine-grained text annotations instead of either alone (ablation) | CLUTCH · 3D-HIW test, text-to-motion | RP3: 0.551 -> 0.721 | Improved |
| increase training data from 7K to 30K sequences (ablation) | CLUTCH · 3D-HIW test, text-to-motion | RP3: 0.513 -> 0.721 | Improved |
Closest papers by meaning
- BOTH2Hands: Inferring 3D Hands from Both Text Prompts and Body Dynamics (CVPR 2024 · similarity 0.88)
- HOLD: Category-agnostic 3D Reconstruction of Interacting Hands and Objects from Video (CVPR 2024 · similarity 0.88)
- How Do I Do That? Synthesizing 3D Hand Motion and Contacts for Everyday Interactions (CVPR 2025 · similarity 0.87)
- HandVQA: Diagnosing and Improving Fine-Grained Spatial Reasoning about Hands in Vision-Language Models (CVPR 2026 · similarity 0.87)
- MEgoHand: Multimodal Egocentric Hand-Object Interaction Motion Generation (NeurIPS 2025 · similarity 0.87)
- The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
