Research map / 3D vision / 3D generation
LiP-Flow: Learning Inference-Time Priors for Codec Avatars via Normalizing Flows in Latent Space
Emre Aksan, Shugao Ma, Akin Caliskan, Stanislav Pidhorskyi, Alexander Richard, Shih-En Wei, Jason Saragih, Otmar Hilliges
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add latent likelihood term L_L to fitting objective (ablation) | LiP-Flow · HMC fitting (one subject) | PSNR: 30.34 -> 33.61 | Improved |
| add prior initialization of latent code (ablation) | LiP-Flow · HMC fitting (one subject) | geometry error: 0.405 -> 0.060 | Improved |
| add prior initialization and L_L together (ablation) | LiP-Flow · HMC fitting (one subject) | PSNR: 28.51 -> 33.61 | Improved |
| add flow cycle in training (ablation) | LiP-Flow · HMC fitting (one subject) | PSNR: 33.07 -> 33.61 | Improved |
| decode samples from P space as well as Q (ablation) | LiP-Flow · HMC fitting (one subject) | PSNR: 33.61 -> 33.03 | Hurt |
| replace KL-divergence with normalizing flow between prior and encoder spaces | LiP-KL-HMC · HMC fitting | PSNR: 34.36 -> 34.98 | Improved |
| replace KL-divergence with normalizing flow | LiP-KL-KPT · Keypoint fitting | geometry error: 0.089 -> 0.053 | Improved |
| replace standard Gaussian prior with learned conditional prior | DAM · HMC fitting | PSNR: 31.65 -> 34.98 | Improved |
| train inference encoder as prior with flow instead of separate regressor | DAM-HMC Reg. · HMC fitting | geometry error: 0.248 -> 0.087 | Improved |
| replace KL with flow (KPT prior) on DAM-encoder reconstruction | LiP-KL-KPT · Reconstruction | PSNR: 35.71 -> 36.22 | Improved |
| train LiP-KL prior jointly with DAM | DAM · Reconstruction | PSNR: 36.05 -> 35.79 | Hurt |
| add unconditional latent flow to DAM (ablation) | DAM · Frontal fitting (one subject) | PSNR: 34.67 -> 35.49 | Improved |
Closest papers by meaning
- LipSync3D: Data-Efficient Learning of Personalized 3D Talking Faces From Video Using Pose and Lighting Normalization (CVPR 2021 · similarity 0.88)
- Generating 3D Faces using Convolutional Mesh Autoencoders (ECCV 2018 · similarity 0.88)
- High-Fidelity Facial Avatar Reconstruction From Monocular Video With Generative Priors (CVPR 2023 · similarity 0.88)
- Synthetic Prior for Few-Shot Drivable Head Avatar Inversion (CVPR 2025 · similarity 0.87)
- LipFormer: High-Fidelity and Generalizable Talking Face Generation With a Pre-Learned Facial Codebook (CVPR 2023 · similarity 0.87)
- DEEPTalk: Dynamic Emotion Embedding for Probabilistic Speech-Driven 3D Face Animation (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
