Research map / 3D vision / 3D generation
Dual Mapping of 2D StyleGAN for 3D-Aware Image Generation and Manipulation (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Lifts a frozen pretrained 2D StyleGAN to 3D-aware generation and manipulation by using two mapping networks that bridge StyleGAN features and a tri-plane 3D representation.
Problem
3D-aware GANs use volume rendering during generator training, which disturbs the semantic disentanglement of StyleGAN latent space and lowers image quality, making attribute manipulation hard.
Approach
Keep the pretrained 2D StyleGAN generator frozen and train two mapping networks: one maps shallow StyleGAN features into a tri-plane 3D feature space, and the other maps resampled tri-plane features at target camera views back into StyleGAN space for the later layers, so 3D consistency is added without altering the latent space.
Key results
- On CelebA-HQ, the method reaches FID 50.96 and CSIM 0.59, versus EG3D's FID 75.33 and CSIM 0.54, with better expression (3.44) and pose (1.04) error.
- On FFHQ, the method reaches FID 58.38 versus EG3D's 83.14, and pose error 1.08 versus StyleGAN2's 1.31.
- Compared with StyleGAN2 on CelebA-HQ, the method has a higher FID (50.96 vs 46.42) but better CSIM (0.59 vs 0.57) and pose error (1.04 vs 1.25).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use dual mapping and tri-plane representation instead of EG3D | 3D-aware image animation · CelebA-HQ | FID: 75.33 -> 50.96 | Helped |
| Use dual mapping and tri-plane representation instead of EG3D | 3D-aware image animation · CelebA-HQ | CSIM: 0.54 -> 0.59 | Helped |
| Use dual mapping and tri-plane representation instead of EG3D | 3D-aware image animation · CelebA-HQ | Exp: 5.81 -> 3.44 | Helped |
| Use dual mapping and tri-plane representation instead of EG3D | 3D-aware image animation · CelebA-HQ | Pose: 1.08 -> 1.04 | Helped |
| Use dual mapping and tri-plane representation instead of EG3D | 3D-aware image animation · FFHQ | FID: 83.14 -> 58.38 | Helped |
| Use dual mapping and tri-plane representation instead of EG3D | 3D-aware image animation · FFHQ | CSIM: 0.52 -> 0.59 | Helped |
| Use dual mapping and tri-plane representation instead of EG3D | 3D-aware image animation · FFHQ | Exp: 6.57 -> 4.70 | Helped |
| Use dual mapping and tri-plane representation instead of EG3D | 3D-aware image animation · FFHQ | Pose: 1.12 -> 1.08 | Helped |
| Add dual mapping and tri-plane view control instead of StyleGAN2 | pretrained 2D StyleGAN2 · CelebA-HQ | FID: 46.42 -> 50.96 | Hurt |
| Add dual mapping and tri-plane view control instead of StyleGAN2 | pretrained 2D StyleGAN2 · CelebA-HQ | CSIM: 0.57 -> 0.59 | Helped |
| Add dual mapping and tri-plane view control instead of StyleGAN2 | pretrained 2D StyleGAN2 · CelebA-HQ | Exp: 3.64 -> 3.44 | Helped |
| Add dual mapping and tri-plane view control instead of StyleGAN2 | pretrained 2D StyleGAN2 · CelebA-HQ | Pose: 1.25 -> 1.04 | Helped |
Surprising findings
- On CelebA-HQ, the proposed method has a worse FID than the base StyleGAN2 (50.96 vs 46.42).
Limitations
- Image quality on CelebA-HQ is slightly below the base StyleGAN2 in FID.
- Evaluation is limited to facial datasets and a single 3D-aware baseline (EG3D).
Experimental setup
Datasets: FFHQ CelebA-HQ
Benchmarks: CelebA-HQ FFHQ
Metrics: FID CSIM expression error head pose error
Baselines: StyleGAN2 EG3D
Builds on: StyleGAN2 EG3D GANSpace
Closest papers by meaning
- Lifting 2D StyleGAN for 3D-Aware Face Generation (CVPR 2021 · similarity 0.91)
- Efficient Geometry-Aware 3D Generative Adversarial Networks (CVPR 2022 · similarity 0.91)
- StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation (CVPR 2022 · similarity 0.90)
- What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs (CVPR 2024 · similarity 0.88)
- Reference-Based 3D-Aware Image Editing with Triplanes (CVPR 2025 · similarity 0.86)
- WarpGAN: Warping-Guided 3D GAN Inversion with Style-Based Novel View Inpainting (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.