Research map / 3D vision / Gaussian splatting
CGS-GAN: 3D Consistent Gaussian Splatting GANs for High Resolution Human Head Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CGS-GAN, a 3D Gaussian splatting GAN for human heads that removes view-conditioning using multi-view regularization, a scalable transformer generator, and a curated FFHQ-Clean dataset, reaching 2048^2 output.
Problem
Existing 3DGS-based 3D GANs for human heads rely on view-conditioning of the latent code, which compromises 3D consistency (identity changes across views), while removing it typically destabilizes GAN training.
Approach
Stabilize training without camera conditioning via multi-view regularization (rendering several views per step and averaging gradients), a point-upsampling generator built on GSGAN that scales linearly to high resolution, replacing the contrastive pose loss with StyleGAN-style conditional discriminator input, random background augmentation, and a filtered FFHQ-Clean dataset.
Key results
- On FFHQ-Clean at 512, CGS-GAN reaches FID 4.53 versus 5.17 for GSGAN and 5.37 for GGHead; at 1024 and 2048 it reaches 5.25 and 7.8.
- Under fixed frontal conditioning, FID_3D for CGS-GAN at 512 on FFHQ is 4.94 versus 10.50 for GSGAN and 7.90 for GGHead.
- Synthesis runs at 152.5 FPS at 512 and 61.3 FPS at 2048 on an RTX 4090; training at 512 uses 15 GB per GPU versus 37 GB for GSGAN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CGS-GAN instead of view-conditioned baselines | 3D head synthesis · FFHQ 512 | FID_3D: 10.50 -> 4.94; 7.90 -> 4.94 | Helped |
| Train on FFHQ-Clean instead of FFHQ | CGS-GAN · 512 resolution | FID: 4.94 -> 4.53 | Helped |
| Train on FFHQ-Clean with full-head crops and rebalanced camera views | CGS-GAN · Steep camera views | rendering quality | Helped |
| Use multi-view regularization (ablation) | CGS-GAN training · Training stability | training stability: about 8M training steps | Helped |
| Render 4 views per training step rather than 2 (ablation) | CGS-GAN multi-view regularization · Training | training performance | Helped |
| Render 4 views per training step rather than 8 (ablation) | CGS-GAN multi-view regularization · Training | training performance | No clear effect |
| Add random colored backgrounds instead of a fixed white background (ablation) | CGS-GAN · Head rendering | hole artifacts | Helped |
| Remove the contrastive loss and use StyleGAN-style discriminator conditioning (ablation) | CGS-GAN · Training convergence | training stability: 3.8 million steps | Helped |
| Remove camera conditioning from the generator and use multi-view training | 3D head synthesis · Novel viewpoints | 3D consistency | Helped |
| Use CGS-GAN instead of GGHead | 3D head synthesis · FFHQ 512 | FID: 4.34 -> 4.94 | Hurt |
| Use CGS-GAN instead of GGHead | 3D head synthesis · FFHQ-Clean 512 | FID: 5.37 -> 4.53 | Helped |
Surprising findings
- Removing view-conditioning from the GAN does not harm convergence when multi-view regularization and the modified loss are used, although prior 3DGS GANs were believed to need it.
- The contrastive pose loss of GSGAN caused training collapse after about 3.8M steps on FFHQ.
Limitations
- Cannot render the back of the head since FFHQ lacks such views; holes appear when rotating beyond 90 degrees.
- Residual occluders and background matting failures (e.g., hats, multiple persons) remain in some outputs.
- High-resolution training is slow, limiting accessibility; no error bars reported.
Experimental setup
Datasets: FFHQ FFHQ-Clean (FFHQC)
Benchmarks: FFHQ FFHQ-Clean
Metrics: FID FID_3D synthesis speed (FPS) training GPU memory training speed
Baselines: GSGAN GGHead EG3D Mimic3D Gaussian Shell Maps
Builds on: GSGAN StyleGAN2 EG3D MODNet
Closest papers by meaning
- What You See is What You GAN: Rendering Every Pixel for High-Fidelity Geometry in 3D GANs (CVPR 2024 · similarity 0.88)
- FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training (NeurIPS 2024 · similarity 0.88)
- Dual Encoder GAN Inversion for High-Fidelity 3D Head Reconstruction from Single Images (NeurIPS 2024 · similarity 0.87)
- GPAvatar: High-fidelity Head Avatars by Learning Efficient Gaussian Projections (CVPR 2025 · similarity 0.87)
- GPS-Gaussian: Generalizable Pixel-wise 3D Gaussian Splatting for Real-time Human Novel View Synthesis (CVPR 2024 · similarity 0.87)
- Human Gaussian Splatting: Real-time Rendering of Animatable Avatars (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.