Research map / Video / Video generation
StickMotion: Generating 3D Human Motions by Drawing a Stickman
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes StickMotion, a diffusion-based text-to-motion model that also accepts user-drawn stickman poses (start/middle/end) as local control, using a Multi-Condition Module and Dynamic Supervision.
Problem
Text-to-motion generation struggles to capture detailed user-imagined body poses from simple text descriptions, and fine-grained control via longer text is burdensome.
Approach
Introduces stickman sketches as a pose condition, generated automatically from dataset joint coordinates (Stickman Generation Algorithm), fused with text via a Multi-Condition Module that computes all condition combinations with cheap feat-decoder offsets instead of masked self-attention, and a Dynamic Supervision loss that lets the model shift each stickman's frame index to a nearby natural posit
Key results
- On HumanML3D, StickMotion reports FID 0.107 (Top1 R-precision 0.518), comparable to ReMoDiffuse (FID 0.103), with better Top1 R-precision.
- On KIT-ML, StickMotion achieves FID 0.141 vs ReMoDiffuse 0.155, with Top1 R-precision 0.430 vs 0.427.
- A user study with 20 volunteers found StickMotion saves about 51.5% of total time (16.4 vs 33.8 units) with a higher user score (8.5 vs 7.3).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Condition Fusion plus Latent Encoder instead of self-attention condition fusion (ablation) | StickMotion · KIT-ML | FID: 0.31 -> 0.14 | Helped |
| Add Latent Encoder to Condition Fusion (ablation) | StickMotion · KIT-ML | FID: 0.18 -> 0.14 | Helped |
| Set the condition-mixture weight to favor text-only output rather than stick-only output (ablation) | StickMotion · KIT-ML | StiSim: 44.8% -> 26.1% | Hurt |
| Increase p(ŵ=w) from 20% to 80% (ablation) | StickMotion · KIT-ML | StiSim: 42.6% -> 51.3% | Helped |
| Include textual description as a generation condition (ablation) | StickMotion · KIT-ML | FID: 4.292 -> 0.164 | Helped |
| Replace detailed text entry with a stickman sketch for motion requests | StickMotion user study · User study | TotalTime: 33.8 -> 16.4 | Helped |
Surprising findings
- Directly assigning stickmen to exact frames degraded naturalness, motivating the flexible index-based dynamic supervision.
- Text dominates performance metrics; the stickman and text conditions were found to have an adversarial relationship in the condition mixture ablation.
Limitations
- Stickman condition may reduce naturalness if placed at exact frames; needs dynamic index adjustment.
- Condition mixture weights require manual tuning (e.g., p(w)=20% biased to stickman), and metrics show trade-offs between text and stickman conditions.
- Evaluation of stickman conditioning relies on synthetic stickmen generated from ground truth rather than real user drawings.
Experimental setup
Datasets: KIT-ML HumanML3D
Benchmarks: HumanML3D test set KIT-ML test set
Metrics: R Precision FID MM Dist Diversity MultiModality StiSim
Baselines: Guo et al. (TEMOS/T2M) MDM MotionDiffuse T2M-GPT ReMoDiffuse
Builds on: DDPM classifier-free diffusion guidance ReMoDiffuse CLIP
Closest papers by meaning
- InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction (NeurIPS 2024 · similarity 0.87)
- AMD: Autoregressive Motion Diffusion (AAAI 2024 · similarity 0.86)
- Text2HOI: Text-guided 3D Motion Generation for Hand-Object Interaction (CVPR 2024 · similarity 0.86)
- Enhanced Fine-Grained Motion Diffusion for Text-Driven Human Motion Synthesis (AAAI 2024 · similarity 0.86)
- HOIAnimator: Generating Text-prompt Human-object Animations using Novel Perceptive Diffusion Models (CVPR 2024 · similarity 0.86)
- InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.