Research map / 3D vision / Face and shape
CtrlAvatar: Controllable Avatars Generation via Disentangled Invertible Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CtrlAvatar, a real-time framework that disentangles avatar deformation into an implicit occupancy geometry driven by an invertible delta network and an explicit, pose-free textured mesh, enabling fast pose-driven animation and 2D-editable clothing.
Problem
Generating drivable, realistic, and editable human avatars from few scans, where existing implicit-field methods must rebuild geometry via Marching Cubes for every pose, making animation slow and topology inconsistent.
Approach
An Invertible Delta Network predicts position-varying conditional offsets on top of LBS so that geometry is queried in a pose-free canonical space (no per-pose mesh extraction), and texture is learned on a fixed pose-free mesh that is deformed by the inverse IDN and LBS, supervised by multi-view 2D image losses.
Key results
- Total pose-driven processing time of 14.33 ms, over 200x faster than XAvatar and nearly 30x faster than XAvatar-NC.
- Texture LPIPS reduced by 0.0812 (16.90%) relative to XAvatar on SX-Humans (0.03993 vs 0.04805).
- Face CD reduced by 0.197 (6.27%) and CD-MAX by 1.629 (8.29%) versus XAvatar on SX-Humans.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CtrlAvatar instead of XAvatar | CtrlAvatar · SX-Humans Face | CD: 3.137 -> 2.940 | Helped |
| use CtrlAvatar instead of XAvatar | CtrlAvatar · SX-Humans Face | CD-MAX: 19.631 -> 18.002 | Helped |
| use CtrlAvatar instead of XAvatar | CtrlAvatar · SX-Humans | LPIPS: 0.04805 -> 0.03993 | Helped |
| use CtrlAvatar instead of XAvatar | CtrlAvatar · SX-Humans | PSNR: 23.602 -> 23.520 | Hurt |
| use CtrlAvatar instead of XAvatar | CtrlAvatar · Avatar generation | time to obtain a new avatar pose: 3352 -> 14.33 ms | Helped |
| remove high-frequency positional encoding (ablation) | CtrlAvatar texture model · Texture ablation | LPIPS: 0.0446 -> 0.0553 (+24.04%) | Hurt |
| remove the deformer (ablation) | CtrlAvatar texture model · Texture ablation | LPIPS: 0.0446 -> 0.0543 (+21.77%) | Hurt |
| remove high-resolution image rendering (ablation) | CtrlAvatar texture model · Texture ablation | LPIPS: 0.0446 -> 0.0507 (+13.68%) | Hurt |
| use only L1 loss (ablation) | CtrlAvatar texture model · Texture ablation | LPIPS: 0.0446 -> 0.0491 (+9.17%) | Hurt |
| use only two viewpoints (ablation) | CtrlAvatar texture model · Texture ablation | LPIPS: 0.0446 -> 0.0473 (+5.71%) | Hurt |
| remove the fusion module (ablation) | CtrlAvatar IDN · IDN ablation | conditional offset accuracy | Hurt |
| add rotation to conditional offset prediction (ablation) | CtrlAvatar IDN · IDN ablation | conditional offset accuracy | Hurt |
Surprising findings
- Adding rotation to the conditional offset prevented the network from correctly training the LBS weights, so translation-only offsets were used.
- Texture PSNR is slightly lower than several baselines (22.91 in the full ablation row, 23.52 vs 23.60 for XAvatar in the main table), which the authors attribute to PSNR favoring smooth data.
Limitations
- Struggles to reconstruct and animate loose clothing such as skirts.
- Editing requires adjusting 2D images; simplified editing process is future work.
Experimental setup
Datasets: X-Humans SX-Humans CustomHumans S-CustomHumans
Benchmarks: SX-Humans S-CustomHumans
Metrics: Chamfer Distance CD-MAX Normal Consistency IoU PSNR SSIM LPIPS inference time (ms)
Baselines: XAvatar XAvatar-NC HAVE-FUN Fast-SNARF Editable-Humans
Builds on: XAvatar SMPL-X Linear Blend Skinning Fast-SNARF
Closest papers by meaning
- DAGSM: Disentangled Avatar Generation with GS-enhanced Mesh (CVPR 2025 · similarity 0.89)
- DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models (CVPR 2024 · similarity 0.88)
- Morphable Diffusion: 3D-Consistent Diffusion for Single-image Avatar Creation (CVPR 2024 · similarity 0.87)
- MPMAvatar: Learning 3D Gaussian Avatars with Accurate and Robust Physics-Based Dynamics (NeurIPS 2025 · similarity 0.87)
- DLCA-Recon: Dynamic Loose Clothing Avatar Reconstruction from Monocular Videos (AAAI 2024 · similarity 0.87)
- FRESA: Feedforward Reconstruction of Personalized Skinned Avatars from Few Images (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.