Research map / 3D vision / Face and shape
Anatomically Constrained Implicit Face Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Anatomical Implicit face Model (AIM), an ensemble of SIREN MLPs that jointly learns an actor's facial anatomy and skin surface so every skin point is derived from an underlying learned anatomy, usable as a drop-in blendshape replacement.
Problem
Actor-specific anatomically constrained face models are state of the art for performance capture and retargeting but are slow to fit and need extensive data capture; anatomical constraints are also only applied sparsely over the face.
Approach
Learn a dense, continuous implicit anatomy surface with soft tissue thickness and normals via coordinate MLPs, reconstruct each skin point as bone point plus thickness times normal, then apply linear blend skinning and corrective expression displacements, with only sparse anatomical regularization; fit by neural reparameterized optimization over frame codes.
Key results
- On 819 frames from 5 sequences of 5 actors, AIM achieves 0.31 mm average fitting error versus 0.51 mm for PBS and 0.83 mm for GBS, with ALM at 0.09 mm.
- Fitting takes about 1 second per frame on an RTX 3090, whereas ALM takes several minutes per frame.
- Model learning takes about 10 minutes on a single RTX 3090 for an actor model with 40,000 vertices and 20 blendshapes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AIM instead of GBS | Actor-specific face fitting · Novel 3D performances | average fitting error (mm): 0.83 -> 0.31 | Helped |
| Use AIM instead of PBS | Actor-specific face fitting · Novel 3D performances | average fitting error (mm): 0.51 -> 0.31 | Helped |
| Use AIM instead of ALM | Actor-specific face fitting · Novel 3D performances | average fitting error (mm): 0.09 -> 0.31 | Hurt |
| Apply expression coefficients globally instead of locally (ablation) | AIM · Novel 3D performances | average fitting error (mm): 0.31 -> 0.86 | Hurt |
Limitations
- Sparse supervision on anatomy causes artifacts on the learned anatomical surface, especially around the lips.
- Skinning does not rigidly deform the anatomy with facial expressions.
- Temporal jitter can occur in fitting for challenging performances if optimization stops early.
Experimental setup
Datasets: studio 3D scans of actors (20 blendshapes per actor) dynamic 3D scans from studio scanner
Benchmarks: in-house actor 3D performance scans (819 frames, 5 sequences, 5 actors)
Metrics: average fitting error (mm)
Baselines: GBS (global blendshapes) PBS (patch blendshapes) ALM (anatomical local model) Chandran et al. 2022 retargeting
Builds on: Anatomical Local Model (Wu et al.) SIREN FLAME Neural reparameterized optimization Local blendshape retargeting (Chandran et al.)
Closest papers by meaning
- High-Quality Face Capture Using Anatomical Muscles (CVPR 2019 · similarity 0.87)
- Implicit Neural Head Synthesis via Controllable Local Deformation Fields (CVPR 2023 · similarity 0.86)
- Parametric Implicit Face Representation for Audio-Driven Facial Reenactment (CVPR 2023 · similarity 0.86)
- ImFace: A Nonlinear 3D Morphable Face Model With Implicit Neural Representations (CVPR 2022 · similarity 0.85)
- RigAnyFace: Scaling Neural Facial Mesh Auto-Rigging with Unlabeled Data (NeurIPS 2025 · similarity 0.85)
- Neural Implicit Morphing of Face Images (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.