Research map / 3D vision / Face and shape

FreeUV: Ground-Truth-Free Realistic Facial UV Texture Recovery via Cross-Assembly Inference Strategy

CVPR 2025 · 3D vision › Face and shape · approach: Face reflectance and relightable rendering · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

FreeUV recovers complete facial UV textures from single in-the-wild face images by training two separate Stable Diffusion ControlNet-style networks without UV ground truth and combining them at inference via Cross-Assembly.

Problem

Recovering high-quality 3D facial UV textures from single-view in-the-wild images is hard because existing methods need costly captured or StyleGAN-synthesized UV ground truth, which limits generalization to makeup, wrinkles, and occlusions.

Approach

Train an appearance network (CLIP-based flaw-tolerant detail extractor with channel attention, masked UV to in-the-wild face) and a structure network (ControlNet, masked 3DMM face to 3DMM UV) independently on non-UV-ground-truth data, then assemble their UV-specific modules into a pretrained Stable Diffusion model at inference for UV-to-UV texture completion.

Key results

• On FFHQ, FreeUV reaches CLIP-I 0.8490, DINO-I 0.7559 and FID 142.39, beating HRN, UV-IDM and the FLAME-based makeup method.
• Ablation: channel attention in the appearance network plus self-attention in the structure network gives the best RMSE 0.0276, SSIM 0.8001, LPIPS 0.0463, PSNR 30.848.
• Inference takes about 4.75 seconds per image with 30 DDIM steps and guidance scale 1.4.

Measured effects reported in the paper

Surprising findings

• Swapping attention designs between the two networks (channel attention in the structure network) degrades alignment, supporting the claim that UV-to-2D and 2D-to-UV mappings need different attention.

Limitations

• Struggles with very fine-grained elements such as accessories, spots and blemishes, which may shift in position or count.
• When regions are occluded (e.g., by a hat), the network may extend surrounding details, reducing localized accuracy.
• Training data filtering was manual, removing images with segmentation errors, hair, or occlusions.

Experimental setup

Datasets: FFHQ CelebAMask-HQ LPFF

Benchmarks: FFHQ CelebAMask-HQ LPFF

Metrics: CLIP-I DINO-I FID RMSE SSIM LPIPS PSNR

Baselines: Deep3Dface FFHQ-UV HRN UV-IDM Makeup Prior Models

Builds on: Stable Diffusion ControlNet Stable-Makeup CLIP

Closest papers by meaning

• OMGTex: One-stage Multi-style Facial Texture Reconstruction without Geometry Guidance (CVPR 2026 · similarity 0.90)
• Towards High-Fidelity 3D Face Reconstruction From In-the-Wild Images Using Graph Convolutional Networks (CVPR 2020 · similarity 0.89)
• FFHQ-UV: Normalized Facial UV-Texture Dataset for 3D Face Reconstruction (CVPR 2023 · similarity 0.89)
• UV-IDM: Identity-Conditioned Latent Diffusion Model for Face UV-Texture Generation (CVPR 2024 · similarity 0.88)
• Blind Face Restoration under Extreme Conditions: Leveraging 3D-2D Prior Fusion for Superior Structural and Texture Recovery (AAAI 2024 · similarity 0.87)
• PSHuman: Photorealistic Single-image 3D Human Reconstruction using Cross-Scale Multiview Diffusion and Explicit Remeshing (CVPR 2025 · similarity 0.85)