Research map / 3D vision / Face and shape
An End-to-End Depth-Based Pipeline for Selfie Image Rectification
Ahmed Alhawwary, Janne Mustaniemi, Phong Nguyen-Ha, Janne Heikkilä
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add end-to-end training of the depth network through the differentiable renderer (ablation) | depth-based rectification pipeline · Synthetic multiview dataset | PSNR: 18.813 -> 19.617 | Improved |
| add end-to-end training of the depth network through the differentiable renderer (ablation) | depth-based rectification pipeline · Synthetic multiview dataset | SSIM: 0.7997 -> 0.8324 | Improved |
| add the horizontal translation (HzT) module (ablation) | pipeline with E2E · Synthetic multiview dataset | PSNR: 19.617 -> 19.894 | Improved |
| add the horizontal translation (HzT) module (ablation) | pipeline with E2E · Synthetic multiview dataset | SSIM: 0.8324 -> 0.8332 | No clear effect |
| use depth-based reprojection pipeline instead of LPUP flow-based pipeline | LPUP · Synthetic Unreal Engine test set | PSNR: 17.80 -> 19.89 | Improved |
| use depth-based reprojection pipeline instead of LPUP flow-based pipeline | LPUP · Synthetic Unreal Engine test set | SSIM: 0.6051 -> 0.8332 | Improved |
| use depth-based pipeline instead of Fried et al. | Fried · CMDP | LPIPS: 0.2128 -> 0.1919 | Improved |
| use depth-based pipeline instead of Disco 3D GAN inversion | Disco · CMDP | landmark error: 0.00448 -> 0.00453 | No clear effect |
| use depth-based pipeline instead of Disco 3D GAN inversion | Disco · CMDP | LPIPS: 0.1934 -> 0.1919 | Improved |
| use depth-based pipeline instead of DFN | DFN · CMDP (smaller subset) | landmark error: 0.00613 -> 0.00499 | Improved |
| use depth-based pipeline instead of Disco inversion for speed | Disco · HD image runtime | runtime: 130 s -> 0.4 s | Improved |
| finetune pretrained depth model on synthetic facial data (ablation) | pretrained monocular depth regressor · synthetic dataset | depth map quality | Improved |
Closest papers by meaning
- Unsupervised Depth Estimation, 3D Face Rotation and Replacement (NeurIPS 2018 · similarity 0.87)
- Learning Perspective Undistortion of Portraits (ICCV 2019 · similarity 0.87)
- InverseFaceNet: Deep Monocular Inverse Face Rendering (CVPR 2018 · similarity 0.86)
- High-Fidelity Monocular Face Reconstruction Based on an Unsupervised Model-Based Face Autoencoder (TPAMI 2018 · similarity 0.86)
- Large Pose 3D Face Reconstruction From a Single Image via Direct Volumetric CNN Regression (ICCV 2017 · similarity 0.86)
- Depth-Aware Generative Adversarial Network for Talking Head Video Generation (CVPR 2022 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
