Research map / 3D vision / NeRF and radiance fields
Artist-Friendly Relightable and Animatable Neural Heads
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends Mixture of Volumetric Primitives (MVP) with an illumination-modulated appearance branch that conditions on per-primitive local light and view directions, trained on interleaved OLAT/full-on multi-view video from a sparse LED-bar rig, to produce relightable and animatable neural heads.
Problem
Creating photorealistic neural head avatars that are both animatable (driven by expression) and relightable under arbitrary lighting, including near-field lights and viewpoints, without a dense light stage.
Approach
Condition the MVP appearance branch on per-primitive local light and view directions (computed from OLAT light position and camera position minus each transformed primitive center), enabling near-field lighting and viewpoints, and train on OLAT sequences captured with a sparse 32-LED-bar setup instead of a dense light stage.
Key results
- On novel light directions, the method reached PSNR 32.19 / 32.20 / 33.73 on subjects S1-S3 vs. 29.68 / 30.13 / 32.37 for re-implemented TRAvatar*.
- On held-out performances, PSNR 39.75 / 29.52 / 31.28 vs. TRAvatar* 28.86 / 28.88 / 31.21 on S1-S3.
- Qualitatively demonstrates relighting under environment maps, near-field point lights, and dolly-zoom camera effects.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the illumination-modulated MVP appearance branch with per-primitive local light and view directions instead of TRAvatar*'s fixed lighting basis | Relightable MVP · Held-out novel light directions, S1–S3 | PSNR: 32.19 vs 29.68; 32.20 vs 30.13; 33.73 vs 32.37 | Helped |
| Use the illumination-modulated MVP appearance branch with per-primitive local light and view directions instead of TRAvatar*'s fixed lighting basis | Relightable MVP · Held-out novel performances, S1–S3 | PSNR: 39.75 vs 28.86; 29.52 vs 28.88; 31.28 vs 31.21 | Helped |
| Use the illumination-modulated MVP appearance branch with per-primitive local light and view directions instead of TRAvatar*'s fixed lighting basis | Relightable MVP · Held-out novel light directions and performances, S1–S3 | PSNR: 28.68 vs 27.30; 29.11 vs 28.13; 30.76 vs 30.65 | Helped |
Limitations
- Only frontal head is modeled; no full 360-degree avatar, with artifacts at hair and neck boundaries.
- Cannot extrapolate to extreme facial expressions outside the training data.
- Gaze and hair motion are not controllable since input is a mesh.
Experimental setup
Datasets: in-house multi-view OLAT and full-on capture of 3 subjects
Benchmarks: held-out OLAT directions, held-out performances, and combined validation sets (in-house)
Metrics: PSNR MAE SSIM LPIPS
Baselines: TRAvatar
Builds on: Mixture of Volumetric Primitives Deep Appearance Model ReNeRF MODNet
Closest papers by meaning
- BecomingLit: Relightable Gaussian Avatars with Hybrid Neural Shading (NeurIPS 2025 · similarity 0.90)
- Relightable and Animatable Neural Avatars from Videos (AAAI 2024 · similarity 0.89)
- Relightable and Animatable Neural Avatar from Sparse-View Video (CVPR 2024 · similarity 0.88)
- Generalizable and Animatable Gaussian Head Avatar (NeurIPS 2024 · similarity 0.87)
- Relightable Gaussian Codec Avatars (CVPR 2024 · similarity 0.87)
- Efficient 3D Implicit Head Avatar with Mesh-anchored Hash Table Blendshapes (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.