Research map / Image generation / Editing and style transfer
Deep CG2Real: Synthetic-to-Real Translation via Image Disentanglement
Sai Bi, Kalyan Sunkavalli, Federico Perazzi, Eli Shechtman, Vladimir G. Kim, Ravi Ramamoorthi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| predict shading and multiply by albedo, with shading discriminator, instead of directly predicting PBR image from OpenGL (ablation) | OpenGL-to-PBR stage · FID O->P | FID: 21.01 -> 7.40 | Improved |
| predict shading (with shading discriminator) rather than PBR image directly from albedo, normal, shading input (ablation) | OpenGL-to-PBR stage · FID O->P | FID: 11.63 -> 7.40 | Improved |
| add albedo, normal and shading buffers as generator input (ablation) | p2pHD · FID O->P | FID: 21.01 -> 11.63 | Improved |
| translate albedo and shading with separate generators instead of image space CycleGAN (ablation) | PBR-to-real stage · FID P->R | FID: 54.80 -> 53.48 | Improved |
| use two-stage pipeline instead of single-stage CycleGAN | OpenGL-to-real translation · FID O->R | FID: 59.42 -> 53.48 | Improved |
| use two-stage pipeline instead of single-stage T2Net | OpenGL-to-real translation · FID O->R | FID: 65.33 -> 53.48 | Improved |
| train normal estimator on PBR-translated images instead of OpenGL images (ablation) | normal estimation network · NYUv2 normal estimation | mean angle error (deg): 37.73 -> 34.82 | Improved |
| add PBR-to-real translation after first stage for normal estimation (ablation) | normal estimation network · NYUv2 normal estimation | mean angle error (deg): 34.82 -> 33.15 | Improved |
| use disentangled PBR-to-real instead of image-space CycleGAN for normal estimation (ablation) | PBR-to-real stage · NYUv2 normal estimation | mean angle error (deg): 33.90 -> 33.15 | Improved |
| use two-stage translation instead of direct OpenGL-to-real CycleGAN for normal estimation | OpenGL-to-real translation · NYUv2 normal estimation | mean angle error (deg): 36.33 -> 33.15 | Improved |
| train depth estimator on PBR-translated images instead of OpenGL images (ablation) | depth estimation network · NYUv2 depth estimation | RMSE: 1.0770 -> 1.0293 | Improved |
| add PBR-to-real translation for depth estimation (ablation) | depth estimation network · NYUv2 depth estimation | RMSE: 1.0293 -> 0.9774 | Improved |
Closest papers by meaning
- Enhancing Photorealism Enhancement (TPAMI 2022 · similarity 0.88)
- T2Net: Synthetic-to-Realistic Translation for Solving Single-Image Depth Estimation Tasks (ECCV 2018 · similarity 0.88)
- Configurable 3D Scene Synthesis and 2D Image Rendering with Per-pixel Ground Truth Using Stochastic Grammars (IJCV 2018 · similarity 0.87)
- Physically-Based Rendering for Indoor Scene Understanding Using Convolutional Neural Networks (CVPR 2017 · similarity 0.86)
- CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Rendering (ECCV 2018 · similarity 0.86)
- OpenRooms: An Open Framework for Photorealistic Indoor Scene Datasets (CVPR 2021 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
