Research map / 3D vision / 3D generation
Controlling the World by Sleight of Hand
Sruthi Sudhakar, Ruoshi Liu, Basile Van Hoorick, Carl Vondrick, Richard Zemel
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| condition on hand masks instead of text captions (TCG) | Stable Diffusion future-frame prediction · SomethingSomethingv2 val subset | SSIM: 0.234 -> 0.414 | Improved |
| add hand-mask conditioning over image-only conditioning (UCG) | Stable Diffusion · SomethingSomethingv2 val subset | PSNR: 12.08 -> 13.72 | Improved |
| use pretrained Stable Diffusion initialization instead of training from scratch (MCVD) | CoSHand · In-the-wild test set | SSIM: 0.373 -> 0.576 | Improved |
| remove Stable Diffusion pretraining (train from scratch) | CoSHand · SomethingSomethingv2 | image quality metrics | Hurt |
| add CLIP image conditioning via cross-attention | CoSHand · SomethingSomethingv2 | all metrics | Improved |
| train on full data instead of 10% | CoSHand · SomethingSomethingv2 | performance and generalization | Improved |
| provide 4 context frames instead of 1 | CoSHand · SomethingSomethingv2 | performance | Improved |
| raise classifier-free guidance scale beyond 2.5 | CoSHand · SomethingSomethingv2 | dynamics of output | Hurt |
| use CFG scale below 2.0 | CoSHand · SomethingSomethingv2 | performance | Hurt |
| use low-quality hand-drawn query masks instead of detailed masks | CoSHand · hand-mask robustness | fine-grained output quality | Mixed |
Closest papers by meaning
- HandWorld: Hand-Centric Unified Video Action Generation (CVPR 2026 · similarity 0.92)
- World Models for Learning Dexterous Hand-Object Interactions from Human Videos (ECCV 2026 · similarity 0.90)
- Vision-Language-Action Pretraining from Large-Scale Human Videos (ICML 2026 · similarity 0.89)
- HandDiffuse: Generative Controllers for Two-Hand Interactions via Diffusion Models (AAAI 2025 · similarity 0.88)
- HandEdit: A Unified Benchmark for Egocentric Human-to-Robot Dexterous Hand Image Editing (NeurIPS 2026 · similarity 0.88)
- G-HOP: Generative Hand-Object Prior for Interaction Reconstruction and Grasp Synthesis (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
