Research map / 3D vision / Face and shape
RGBAvatar: Reduced Gaussian Blendshapes for Online Modeling of Head Avatars
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RGBAvatar, which maps tracked FLAME parameters via an MLP to a compact set of learned Gaussian blendshapes, plus color initialization, batch-parallel rasterization, and a local-global sampling strategy for fast online head-avatar reconstruction.
Problem
Reconstructing photorealistic animatable head avatars from monocular video quickly enough for on-the-fly use, since existing Gaussian blendshape methods rely on many fixed 3DMM bases and slow training.
Approach
Replace fixed 3DMM linear blendshape weights with an MLP that maps tracked FLAME parameters to reduced weights over learnable, subject-adaptive Gaussian blendshape bases (K=20), combined with fast color initialization, batch-parallel CUDA-stream rasterization, and a local-global replay sampling scheme for streaming data.
Key results
- Achieves best PSNR on most of 12 subjects (e.g., 33.89 on INSTA bala vs 33.21 for GaussianBlendShapes) with only 20 blendshapes versus 50-80 for GaussianBlendShapes.
- Reconstructs an avatar in about 81 seconds on RTX 3090 at 398 FPS rendering, versus 20 min for GaussianBlendShapes.
- Training throughput of 630 images/s with 100% GPU utilization, versus 151 without batch-parallel rasterization; online variant reaches quality close to offline (e.g., PSNR 30.86 vs offline-level results in ablation).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use reduced, subject-adaptive Gaussian blendshapes | RGBAvatar · INSTA and GaussianBlendShapes datasets | PSNR | Helped |
| add LPIPS loss with weight 0.05 | RGBAvatar · INSTA and GaussianBlendShapes datasets | LPIPS | Helped |
| use online reconstruction | RGBAvatar · INSTA and GaussianBlendShapes datasets | PSNR | Mixed |
| remove the global sampling pool (ablation) | RGBAvatar online reconstruction · INSTA dataset | PSNR: 30.26 -> 30.86 | Hurt |
| remove the local sampling pool (ablation) | RGBAvatar online reconstruction · INSTA dataset | PSNR: 30.46 -> 30.86 | Hurt |
| remove batch-parallel rasterization (ablation) | RGBAvatar training · Training | Throughput: 151 -> 630 | Hurt |
| use naive batch-parallel rasterization instead of the optimized method (ablation) | RGBAvatar training · Training | Throughput: 463 -> 630 | Hurt |
| map full FLAME parameters to reduced blendshape weights with an MLP rather than directly using truncated FLAME expression coefficients (ablation) | RGBAvatar · Self-reenactment | Reconstruction quality | Helped |
| use 20 reduced blendshapes rather than more bases (ablation) | RGBAvatar · INSTA dataset | PSNR | No clear effect |
| add color initialization for Gaussian colors (ablation) | RGBAvatar · Training convergence | Loss | Helped |
Surprising findings
- Reduced blendshapes learned by an MLP outperform truncated raw FLAME expression coefficients even with fewer bases (20 vs 50).
Limitations
- Visible artifacts under large novel-pose rendering far from training poses.
- Gaussians stay bound to their initial mesh triangles, relying on the 3DMM prior.
- Potential misuse for deepfakes is acknowledged.
Experimental setup
Datasets: INSTA dataset GaussianBlendShapes dataset
Benchmarks: INSTA dataset GaussianBlendShapes dataset
Metrics: PSNR SSIM LPIPS training time rendering FPS training throughput GPU utilization
Baselines: SplattingAvatar GaussianAvatars FlashAvatar MonoGaussianAvatar GaussianBlendShapes
Builds on: 3D Gaussian Splatting Gaussian Blendshapes FlashAvatar FLAME
Closest papers by meaning
- HRAvatar: High-Quality and Relightable Gaussian Head Avatar (CVPR 2025 · similarity 0.92)
- GPAvatar: High-fidelity Head Avatars by Learning Efficient Gaussian Projections (CVPR 2025 · similarity 0.91)
- Generalizable and Animatable Gaussian Head Avatar (NeurIPS 2024 · similarity 0.91)
- HERA: Hybrid Explicit Representation for Ultra-Realistic Head Avatars (CVPR 2025 · similarity 0.90)
- FlashAvatar: High-fidelity Head Avatar with Efficient Gaussian Embedding (CVPR 2024 · similarity 0.90)
- GaussianAvatars: Photorealistic Head Avatars with Rigged 3D Gaussians (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.