Research map / 3D vision / Face and shape
Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Rendering
Yuxuan Zhang, Wenzheng Chen, Huan Ling, Jun Gao, Yinan Zhang, Antonio Torralba, Sanja Fidler
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the inverse graphics network on StyleGAN-generated multi-view data instead of Pascal3D data | Inverse graphics network · Pascal3D test | 2D IOU: 0.80 -> 0.76 | Hurt |
| Train the inverse graphics network on StyleGAN-generated multi-view data instead of Pascal3D data | Inverse graphics network · StyleGAN test | 2D IOU: 0.81 -> 0.95 | Improved |
| Train on the StyleGAN-generated dataset rather than Pascal3D | Inverse graphics network · Pascal3D test | overall user preference: 25.9% -> 57.5% | Improved |
| Train on the StyleGAN-generated dataset rather than Pascal3D | Inverse graphics network · Pascal3D test | shape user preference: 26.4% -> 61.6% | Improved |
| Train on the StyleGAN-generated dataset rather than Pascal3D | Inverse graphics network · Pascal3D test | texture user preference: 32.8% -> 56.3% | Improved |
| Use multi-view consistency loss during training (ablation) | Inverse graphics network · 3D reconstruction | shape and texture quality | Improved |
| Use perceptual loss during training (ablation) | Inverse graphics network · 3D reconstruction | texture realism | Improved |
| Fine-tune StyleGAN jointly with the mapping network (ablation) | StyleGAN-R · 3D neural rendering | consistency of shape, texture and background | Improved |
| Initialize cameras with coarse view bins instead of keypoints and SfM | Inverse graphics network · StyleGAN test | 2D IOU: 0.953 -> 0.952 | No clear effect |
| Optimize the original StyleGAN latent code directly for image reconstruction instead of using the mapping network and fine-tuned StyleGAN-R | StyleGAN image editing · Viewpoint control | reconstruction quality | Hurt |
| Use the mapping network without fine-tuning StyleGAN (ablation) | StyleGAN neural renderer · 3D neural rendering | consistency of shape, texture and background | Hurt |
Closest papers by meaning
- High-Fidelity 3D GAN Inversion by Pseudo-Multi-View Optimization (CVPR 2023 · similarity 0.91)
- Inferring 3D Shapes from Image Collections Using Adversarial Networks (IJCV 2020 · similarity 0.89)
- Learning Generative Models of Textured 3D Meshes From Real-World Images (ICCV 2021 · similarity 0.89)
- What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs (CVPR 2024 · similarity 0.88)
- In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editing (CVPR 2024 · similarity 0.88)
- Dual Encoder GAN Inversion for High-Fidelity 3D Head Reconstruction from Single Images (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
