Research map / 3D vision / Face and shape
StrokeFaceNeRF: Stroke-based Facial Appearance Editing in Neural Radiance Field
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes StrokeFaceNeRF, a color-stroke-based editing method for EG3D facial NeRFs that encodes stroke maps into the latent space with a transformer fusion module, a geometry density loss, and 3D volumetric mask fusion for local edits.
Problem
Editing facial NeRF appearance precisely and interactively: existing 3D-aware facial NeRF methods control appearance via text, lighting, or reference images, and 2D stroke-based editors do not preserve geometry or transfer to 3D radiance fields.
Approach
Encode user color-stroke maps into EG3D's latent space, fuse them with the original latent via a cross-attention transformer to keep geometry, add a density loss that keeps 3D surface density consistent, and lift 2D masks into a 3D mask volume for blending original and edited triplane features.
Key results
- Ours achieves FID 16.86 and KID 0.83 vs. best baseline pSp-mixing FID 31.32 and KID 2.00 in the comparison table.
- E-SSIM edge-preservation in edited regions improves to 84.30 vs. 81.50 for pSp-mixing.
- Ablation: removing the geometry density loss raises FID from 16.86 to 21.44 and lowers E-SSIM from 84.30 to 77.35.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add geometry density loss during training (ablation) | StrokeFaceNeRF · EG3D randomly sampled data | E-SSIM ×102: 77.35 -> 84.30 | Helped |
| add geometry density loss during training (ablation) | StrokeFaceNeRF · EG3D randomly sampled data | ID: 0.67 ± 0.22 -> 0.85± 0.10 | Helped |
| replace geometry density loss with depth image loss (ablation) | StrokeFaceNeRF · EG3D randomly sampled data | E-SSIM ×102: 84.30 -> 80.10 | Hurt |
| use volumetric local mask fusion (ablation) | StrokeFaceNeRF · EG3D randomly sampled data | E-SSIM ×102: 82.97 -> 84.30 | Helped |
| use volumetric local mask fusion (ablation) | StrokeFaceNeRF · EG3D randomly sampled data | FID: 28.33 -> 16.86 | Helped |
Surprising findings
- Directly encoding stroke maps with pSp-style encoders changes geometry and background even with style-mixing, motivating the latent fusion design.
Limitations
- Struggles to edit highly detailed regions such as eyeballs.
- Overly complex multi-color strokes (e.g., rainbow hair) get averaged toward a single color due to training data limits.
- Evaluation uses GAN-based metrics and an identity score; no user study is reported.
Experimental setup
Datasets: CelebA-HQ FFHQ EG3D random samples
Benchmarks: FFHQ (1000 generated editing examples) EG3D random samples (ablation)
Metrics: FID KID ID consistency E-SSIM
Baselines: SC-FEGAN SDEdit pSp pSp-mixing
Builds on: EG3D pSp PTI StyleGAN2 BiSeNet
Closest papers by meaning
- FFaceNeRF: Few-shot Face Editing in Neural Radiance Fields (CVPR 2025 · similarity 0.90)
- InstructPix2NeRF: Instructed 3D Portrait Editing from a Single Image (ICLR 2024 · similarity 0.88)
- LAENeRF: Local Appearance Editing for Neural Radiance Fields (CVPR 2024 · similarity 0.87)
- Text-Guided 3D Face Synthesis - From Generation to Editing (CVPR 2024 · similarity 0.86)
- 3D-Aware Face Editing via Warping-Guided Latent Direction Learning (CVPR 2024 · similarity 0.86)
- Neural 3D Strokes: Creating Stylized 3D Scenes with Vectorized 3D Strokes (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.