Research map / 3D vision / Face and shape
BecomingLit: Relightable Gaussian Avatars with Hybrid Neural Shading
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces BecomingLit, a relightable 3D Gaussian head avatar with hybrid neural shading (neural diffuse BRDF plus analytical Cook-Torrance specular), trained on a new low-cost multi-view OLAT light stage dataset.
Problem
Reconstructing photorealistic, relightable head avatars that can be animated and rendered at interactive rates, which requires disentangled material and lighting, is hard because existing light stage setups are costly and public relighting datasets are scarce.
Approach
Combine expression-dependent 3D Gaussians animated by FLAME with a hybrid shading model: a small shared MLP learns view-independent diffuse light transport from spherical harmonics of the light, while specular reflection uses an analytical Cook-Torrance term with a Blinn-Phong lobe mixture and learned view-dependent normal offsets.
Key results
- On held-out light relighting, the method reaches 31.38 PSNR / 0.8956 SSIM / 0.1040 LPIPS versus 29.21 / 0.8462 / 0.1659 for RGCA.
- On relighting plus self-reenactment, it reaches 28.08 PSNR / 0.8730 SSIM / 0.1317 LPIPS versus 26.31 / 0.8206 / 0.1917 for RGCA.
- Full avatar renders at 17 ms per frame (1100x1604) on an RTX A6000, versus 20 ms for RGCA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BecomingLit instead of RGCA_FLAME | BecomingLit avatar reconstruction · BecomingLit held-out lights, relighting | PSNR: 29.78 -> 31.38 | Helped |
| Use BecomingLit instead of RGCA_FLAME | BecomingLit avatar reconstruction · BecomingLit held-out lights, relighting + self-reenactment | LPIPS: 0.1667 -> 0.1317 | Helped |
| Replace hybrid neural shading with Lambertian diffuse and Cook-Torrance specular (ablation) | BecomingLit · Relighting and relighting + self-reenactment | PSNR: 31.38 -> 29.42; 28.08 -> 26.31 | Hurt |
| Replace neural diffuse shading with learned PRT diffuse (ablation) | BecomingLit · Relighting and relighting + self-reenactment | PSNR: 31.38 -> 29.23; 28.08 -> 25.47 | Hurt |
| Replace Cook-Torrance specular with a Spherical Gaussian term (ablation) | BecomingLit · Relighting and relighting + self-reenactment | PSNR: 31.38 -> 31.55; 28.08 -> 28.09 | Mixed |
| Add alpha loss using foreground segmentation masks (ablation) | BecomingLit · Environment map relighting | visual artifacts | Helped |
| Add expression-dependent features (ablation) | BecomingLit · Relighting and self-reenactment | PSNR: 31.23 -> 31.38 | Helped |
Surprising findings
- RGCA conditioned on FLAME expression parameters performed better than RGCA with its learned per-identity expression space, which the authors attribute to per-subject expression spaces being suboptimal for reenactment.
- On Goliath-4, RGCA slightly beats the method on PSNR (29.89 vs 29.70 full cam set; 28.74 vs 28.68 subset), with the method winning only on SSIM and LPIPS.
Limitations
- Requires several thousand frames and diverse training expressions; cannot yet create avatars from casual phone captures with uncalibrated lighting.
- Inherits FLAME limitations: limited expressiveness, sensitive to gaze tracking failures, and no mouth interior modeling.
- Neural diffuse shading is trained from scratch per avatar, with no learned appearance prior.
Experimental setup
Datasets: BecomingLit OLAT light stage dataset Goliath-4 VFHQ
Benchmarks: BecomingLit OLAT dataset (held-out lights and expressions) Goliath-4
Metrics: PSNR SSIM LPIPS runtime (ms)
Baselines: RGCA RGCA (FLAME expression variant) PBR shading ablation PRT diffuse ablation Spherical Gaussian specular ablation
Builds on: 3D Gaussian Splatting Relightable Gaussian Codec Avatars FLAME Cook-Torrance split-sum approximation
Closest papers by meaning
- Relightable and Animatable Neural Avatars from Videos (AAAI 2024 · similarity 0.90)
- Artist-Friendly Relightable and Animatable Neural Heads (CVPR 2024 · similarity 0.90)
- Relightable Gaussian Codec Avatars (CVPR 2024 · similarity 0.89)
- Generalizable and Animatable Gaussian Head Avatar (NeurIPS 2024 · similarity 0.88)
- HRAvatar: High-Quality and Relightable Gaussian Head Avatar (CVPR 2025 · similarity 0.88)
- GPAvatar: High-fidelity Head Avatars by Learning Efficient Gaussian Projections (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.