Research map / 3D vision / 3D generation
HyPlaneHead: Rethinking Tri-plane-like Representations in Full-Head Image Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HyPlaneHead, a 3D-aware GAN for full-head synthesis using a hybrid planar-plus-spherical "hy-plane" representation, near-equal-area square-to-sphere warping, and a unify-split strategy that generates a single-channel feature map to avoid cross-channel feature penetration.
Problem
Tri-plane-like representations in 3D-aware GANs suffer from feature entanglement causing mirroring artifacts, uneven (non-equal-area) warping in spherical tri-planes causing seam/polar artifacts and poor feature-map utilization, and feature penetration across convolutional channels between planes.
Approach
Combine planar and spherical feature planes in one hy-plane representation, warp the spherical plane with a Lambert azimuthal equal-area projection plus elliptical grid mapping, and generate one unified single-channel feature map that is split spatially into planes so planes do not share channels.
Key results
- Hy-plane (3+1) with area-biased splitting and near-equal-area warping reaches FID 8.14 vs 9.22 for EG3D tri-plane and 8.64 for SphereHead spherical tri-plane (50K samples).
- Hy-plane (2+2) with area-biased splitting achieves the best FID-random of 9.84, vs 11.23 for EG3D and 10.71 for SphereHead.
- Replacing tri-plane with hy-plane adds about 5.5% training time and 3.8% inference time per the authors' measurements.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a spherical plane to tri-plane representation | Full-head Image Synthesis · Full-head Image Synthesis | FID: 9.22 -> 8.54 | Helped |
| Apply unify-split to tri-plane (ablation) | Tri-plane · Full-head Image Synthesis | FID: 9.22 -> 8.86 | Helped |
| Apply unify-split to tri-plane (ablation) | Tri-plane · Full-head Image Synthesis | FID-random: 11.23 -> 11.52 | Hurt |
| Apply unify-split to spherical tri-plane (ablation) | Spherical Tri-plane · Full-head Image Synthesis | FID: 8.64 -> 8.36 | Helped |
| Apply unify-split to spherical tri-plane (ablation) | Spherical Tri-plane · Full-head Image Synthesis | FID-random: 10.71 -> 10.42 | Helped |
| Output two spherical tri-planes simultaneously from a shared branch (ablation) | Dual Spherical Tri-plane · Full-head Image Synthesis | FID: 8.68 -> 11.9 | Hurt |
| Output two spherical tri-planes simultaneously from a shared branch (ablation) | Dual Spherical Tri-plane · Full-head Image Synthesis | FID-random: 10.28 -> 13.54 | Hurt |
| Add unify-split to Hy-plane (3+1) (ablation) | Hy-plane (3+1) · Full-head Image Synthesis | FID: 8.54 -> 8.31 | Helped |
| Add unify-split to Hy-plane (3+1) (ablation) | Hy-plane (3+1) · Full-head Image Synthesis | FID-random: 10.66 -> 10.18 | Helped |
| Add near-equal-area wrapping to evenly split Hy-plane (3+1) (ablation) | Hy-plane (3+1) · Full-head Image Synthesis | FID: 8.31 -> 8.18 | Helped |
| Add near-equal-area wrapping to evenly split Hy-plane (3+1) (ablation) | Hy-plane (3+1) · Full-head Image Synthesis | FID-random: 10.18 -> 9.96 | Helped |
| Use area-biased rather than even splitting (ablation) | Wrapped Hy-plane (3+1) · Full-head Image Synthesis | FID: 8.18 -> 8.14 | Helped |
Surprising findings
- Applying the unify-split strategy to the plain tri-plane lowers FID but raises FID-random, which the authors attribute to exacerbated back-side mirroring.
- Directly outputting two spherical tri-planes (dual spherical tri-plane) yields much worse FID (11.9) than the single-channel unify-split variant, indicating cross-plane interference.
Limitations
- Visual flickering or instability in fine details when rendering from gradually changing viewpoints, attributed to the GAN backbone.
- Struggles with complex hairstyles such as ponytails and braids.
- Slightly slower inference than baseline methods and a need for higher-resolution output feature maps.
Experimental setup
Datasets: FFHQ CelebA LPFF WildHead K-Hairstyle in-house 6K large-pose head images
Benchmarks: FFHQ-based full-head image synthesis (FID, FID-random) single-view GAN inversion with PTI
Metrics: FID FID-random parameter count training speed inference speed VRAM usage
Baselines: EG3D PanoHead SphereHead Tri-plane (retrained) Tri-grid Dual Spherical Tri-plane OrthoPlanes
Builds on: EG3D SphereHead PanoHead StyleGAN2 Pivotal Tuning Inversion
Closest papers by meaning
- Dual Encoder GAN Inversion for High-Fidelity 3D Head Reconstruction from Single Images (NeurIPS 2024 · similarity 0.85)
- AirPlanes: Accurate Plane Estimation via 3D-Consistent Embeddings (CVPR 2024 · similarity 0.85)
- Dual Mapping of 2D StyleGAN for 3D-Aware Image Generation and Manipulation (Student Abstract) (AAAI 2024 · similarity 0.85)
- G-NeRF: Geometry-enhanced Novel View Synthesis from Single-View Images (CVPR 2024 · similarity 0.84)
- World Tracing: Pixel-Aligned Geometry Beyond the Visible (NeurIPS 2026 · similarity 0.84)
- Neural Processing of Tri-Plane Hybrid Neural Fields (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.