Research map / 3D vision / 3D generation
DRiVE: Diffusion-based Rigging Empowers Generation of Versatile and Expressive Characters
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DRiVE, a pipeline that generates 3D Gaussian characters from a single image or text and rigs them with skeletons (including hair and clothing) and skinning using a diffusion model (GSDiff), trained on a new 9420-mesh dataset AnimeRig.
Problem
Animating generated 3D characters with heterogeneous structures such as garments and hair is hard due to a lack of large-scale rigging datasets and effective rigging methods for 3D Gaussian representations.
Approach
Treat joint positions as spatial distributions denoised by a conditional diffusion model conditioned on 3D Gaussian means (via kNN geometric features) and CLIP features of multi-view renderings, instead of regressing joints; then connect bones via MST on BoneFlow and predict skinning from k-NN-initialized weights.
Key results
- Skeleton IoU of 70.48% vs 28.69% for RigNet (ground-truth mesh input) on the AnimeRig test set.
- Skinning precision 78.34% and average L1 0.48, versus 44.48% precision and 0.88 L1 for GeoVoxel.
- Diffusion-based joint prediction outperforms the regression-based counterpart (IoU 70.48% vs 46.72%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use conditional diffusion instead of regression for joint estimation | DRiVE joint estimation · AnimeRig test set | IoU: 46.72% -> 70.48% | Helped |
| add 3D Gaussian geometric conditioning (ablation) | DRiVE joint estimation · AnimeRig test set | IoU: 56.56% -> 70.48% | Helped |
| add local 3D Gaussian conditioning (ablation) | DRiVE joint estimation · AnimeRig test set | IoU: 58.71% -> 70.48% | Helped |
| add image conditioning (ablation) | DRiVE joint estimation · AnimeRig test set | IoU: 67.14% -> 70.48% | Helped |
| use textured mesh input instead of 3D Gaussian (ablation) | DRiVE joint estimation · AnimeRig test set | IoU: 70.48% -> 71.05% | Helped |
| use plain mesh input instead of 3D Gaussian (ablation) | DRiVE joint estimation · AnimeRig test set | IoU: 70.48% -> 67.90% | Hurt |
| add rendered image input (ablation) | DRiVE skinning estimation · AnimeRig test set | avg L1: 0.69 -> 0.48 | Helped |
| add initial skinning values Sinit (ablation) | DRiVE skinning estimation · AnimeRig test set | avg L1: 0.65 -> 0.48 | Helped |
| add smoothness loss (ablation) | DRiVE skinning estimation · AnimeRig test set | avg L1: 0.49 -> 0.48 | Helped |
Surprising findings
- Training RigNet on the new dataset, its networks struggled to converge with the more complex skeletal structures.
Limitations
- Skeleton node density is deterministic and cannot be tuned by user preference.
- No mature collision detection for 3D Gaussians, so rigged characters can self-intersect (e.g., leg and dress).
Experimental setup
Datasets: AnimeRig
Benchmarks: AnimeRig test set
Metrics: IoU Precision Recall CD-J2J CD-J2B CD-B2B avg L1 skinning error
Baselines: AnimSkelVolNet RigNet GeoVoxel CharacterGen Regression-based joint predictor
Builds on: LGM RigNet TARig CharacterGen SV3D
Closest papers by meaning
- HumanRig: Learning Automatic Rigging for Humanoid Character in a Large Scale Dataset (CVPR 2025 · similarity 0.88)
- Make-It-Animatable: An Efficient Framework for Authoring Animation-Ready 3D Characters (CVPR 2025 · similarity 0.87)
- SimAvatar: Simulation-Ready Avatars with Layered Hair and Clothing (CVPR 2025 · similarity 0.87)
- TapMo: Shape-aware Motion Generation of Skeleton-free Characters (ICLR 2024 · similarity 0.87)
- MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics (NeurIPS 2025 · similarity 0.86)
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.