Research map / Image generation / Editing and style transfer
Neural Implicit Morphing of Face Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IFMorph, a face morphing method that represents time-dependent warpings and blendings of face images with coord-based sinusoidal MLPs trained via energy functionals (identity/inverse, landmark data, thin-plate, Poisson-style gradient blending), optionally combined with diffAE generative blending.
Problem
Face morphing requires feature alignment (warping) and seamless blending between faces that differ in pose, lighting, gender, and ethnicity; classical discrete methods and generative latent interpolation lack temporal coherence and exact alignment.
Approach
Model the warping as a single time-dependent sinusoidal MLP that encodes both direct and inverse transformations (so no inversion is needed), train it with closed-form derivative-based energy terms (thin-plate regularization, identity/inverse constraints, landmark data term), and blend the aligned images linearly, in the gradient domain via a second neural network, or generatively in diffAE latent
Key results
- Ours (seamless mix) achieves FID 22.604 vs. 35.653 for StyleGAN3 and 41.356 for diffAE on 1220 FRLL morphings.
- Ours (linear blending) LPIPS (I, I1) of 0.164 is comparable to generative baselines (0.173 StyleGAN3, 0.186 diffAE).
- The warping uses a single-hidden-layer sinusoidal MLP with 128 neurons, which is compact.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| neural warping (time-dependent coord-based MLP alignment) with seamless mix blending instead of OpenCV warping | face morphing · FRLL morphings, t=0.5 | FID: 68.234 -> 22.604 | Helped |
| neural warping with linear blending vs. OpenCV warping | face morphing · FRLL morphings, t=0.5 | FID: 68.234 -> 31.950 | Helped |
| neural warping with seamless mix blending vs. StyleGAN3 generative morphing | face morphing · FRLL morphings, t=0.5 | FID: 35.653 -> 22.604 | Helped |
| neural warping with diffAE generative blending | face morphing · FRLL morphings, t=0.5 | FID: 41.356 -> 40.224 (diffAE alone to combined, marginal) | Hurt |
| seamless mix blending (vs. linear blending) on neural warping (ablation) | neural warping morphing · FRLL morphings, t=0.5 | LPIPS (I0, I): 0.158 -> 0.081 | Helped |
| seamless mix blending (vs. linear blending) on neural warping (ablation) | neural warping morphing · FRLL morphings, t=0.5 | LPIPS (I, I1): 0.164 -> 0.241 | Hurt |
| thin-plate energy regularization in neural warping training (ablation) | neural warping · qualitative ablation (loss term impact) | alignment quality (qualitative) | Helped |
| data constraint removed from neural warping loss (ablation) | neural warping · qualitative ablation (loss term impact) | feature alignment (qualitative) | Hurt |
| warping (identity/inverse) constraint W in loss (ablation) | neural warping · qualitative ablation (loss term impact) | warping quality (qualitative) | No clear effect |
Surprising findings
- The thin-plate energy constraint was found to be crucial, while the warping (identity/inverse) constraint had less influence, mostly acting on finer details.
Limitations
- Learning non-invertible transformations (e.g., collapsing regions) can lead to inconsistencies in the inverse/direct warp.
- Generative blending with diffAE requires FFHQ alignment for baselines and blurs reconstructions; diffAE blending scored worse on FID (40.224).
- Evaluated on a single dataset (FRLL) with morphings of similar faces only.
Experimental setup
Datasets: FRLL
Benchmarks: FRLL morphings
Metrics: FID LPIPS
Baselines: OpenCV morphing StyleGAN3 diffAE MorDiff
Builds on: Poisson image editing SIREN diffAE thin-plate splines
Closest papers by meaning
- FLOWING: Implicit Neural Flows for Structure-Preserving Morphing (NeurIPS 2025 · similarity 0.86)
- ControlFace: Harnessing Facial Parametric Control for Face Rigging (CVPR 2025 · similarity 0.85)
- Generalizable Face Landmarking Guided by Conditional Face Warping (CVPR 2024 · similarity 0.85)
- Anatomically Constrained Implicit Face Models (CVPR 2024 · similarity 0.84)
- Pose Modulated Avatars from Video (ICLR 2024 · similarity 0.84)
- RigAnyFace: Scaling Neural Facial Mesh Auto-Rigging with Unlabeled Data (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.