Research map / 3D vision / Gaussian splatting
AniGS: Animatable Gaussian Avatar from a Single Image with Inconsistent Gaussian Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates multi-view canonical-pose RGB and normal images from a single human photo with a fine-tuned CogVideo video diffusion transformer, then reconstructs an animatable Gaussian avatar via a 4D Gaussian Splatting model that treats view inconsistencies as temporal variation.
Problem
Creating animatable 3D human avatars from a single in-the-wild image: reconstruction methods lose fine detail and generative video methods are view-inconsistent and slow.
Approach
Adapt a reference-image and SMPL-X-conditioned video diffusion transformer (CogVideo-2B) to jointly produce multi-view canonical RGB and normal images, pretrained on in-the-wild videos, and reconstruct from the inconsistent views by fitting a 4D Gaussian Splatting model whose canonical space is the avatar.
Key results
- Canonical multi-view generation LPIPS 0.1023 and normal loss 0.101 vs CHAMP* PSNR 20.025, outperforming it on PSNR-LPIPS, with MagicMan+ having higher PSNR (23.775) but worse LPIPS.
- Single-image canonical reconstruction LPIPS 0.1085, CLIP 90.370, FID 77.879, outperforming CharacterGen, En3D, LGM, MagicMan and SiTH.
- Animation PSNR 21.475 / SSIM 0.857 / LPIPS 0.137, beating En3D (19.244) and CharacterGen (17.570); 4DGS improves reconstruction FID from 114.645 to 77.879 over 3DGS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AniGS instead of CHAMP* for multi-view canonical image generation | Multi-view canonical image generation · Multi-view canonical image generation | PSNR: 20.025 -> 23.125 | Helped |
| use AniGS instead of CHAMP* for multi-view canonical image generation | Multi-view canonical image generation · Multi-view canonical image generation | LPIPS: 0.1458 -> 0.1023 | Helped |
| use AniGS instead of En3D for avatar reconstruction | Human4DiT synthetic datasets · Human4DiT synthetic datasets | LPIPS: 0.1576 -> 0.1085 | Helped |
| use AniGS instead of En3D for animation | Human animation · Human animation | PSNR: 19.244 -> 21.475 | Helped |
| replace 3DGS with 4DGS reconstruction (ablation) | AniGS reconstruction · Human4DiT synthetic datasets, reconstruction | FID: 114.645 -> 77.879 | Helped |
| replace 3DGS with 4DGS reconstruction (ablation) | AniGS reconstruction · Human4DiT synthetic datasets, animation | PSNR: 20.313 -> 21.475 | Helped |
| add normal regularization (ablation) | AniGS 4DGS reconstruction · Novel-pose avatar animation | surface detail and noise | Helped |
| add anisotropic regularization (ablation) | AniGS 4DGS reconstruction · Novel-pose avatar animation | spike artifacts | Helped |
| initialize 4DGS with a coarse mesh instead of random points (ablation) | AniGS 4DGS reconstruction · Avatar reconstruction | reconstruction quality | Helped |
Surprising findings
- MagicMan+ achieves higher PSNR than the proposed generator (23.775 vs 23.125) despite worse LPIPS and normal error.
Limitations
- Still requires several minutes of optimization per avatar, not feed-forward
- Struggles with half-body image inputs
- Multi-view inconsistencies from the video generator still require 4DGS handling
Experimental setup
Datasets: in-the-wild human videos (~100K) 2K2K THuman2.0 THuman2.1 CustomHumans Thuman Wind (Thwindom) RenderPeople Human4DiT synthetic rigged avatars
Benchmarks: Human4DiT synthetic avatars (50 rigged avatars)
Metrics: PSNR SSIM LPIPS FID CLIP score normal loss user study ranking
Baselines: MagicMan CHAMP CharacterGen En3D SiTH LGM
Builds on: CogVideo 3D Gaussian Splatting CHAMP 4D Gaussian Splatting
Closest papers by meaning
- 3DGS-Avatar: Animatable Avatars via Deformable 3D Gaussian Splatting (CVPR 2024 · similarity 0.92)
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.90)
- GASP: Gaussian Avatars with Synthetic Priors (CVPR 2025 · similarity 0.90)
- Animatable Gaussians: Learning Pose-dependent Gaussian Maps for High-fidelity Human Avatar Modeling (CVPR 2024 · similarity 0.89)
- ASH: Animatable Gaussian Splats for Efficient and Photoreal Human Rendering (CVPR 2024 · similarity 0.89)
- 3D²-Actor: Learning Pose-Conditioned 3D-Aware Denoiser for Realistic Gaussian Avatar Modeling (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.