Research map / 3D vision / NeRF and radiance fields
Taming Latent Diffusion Model for Neural Radiance Field Inpainting
Chieh Hubert Lin, Changil Kim, Jia-Bin Huang, Qinbo Li, Chih-Yao Ma, Johannes Kopf, Ming-Hsuan Yang, Hung-Yu Tseng
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add masked adversarial loss instead of pixel reconstruction loss in inpainting region (ablation) | MALD-NeRF · SPIn-NeRF | FID: 305.60 -> 183.25 | Improved |
| add masked adversarial loss instead of LPIPS loss (ablation) | MALD-NeRF · SPIn-NeRF | FID: 192.86 -> 183.25 | Improved |
| add LPIPS loss on top of adversarial training (ablation) | MALD-NeRF · SPIn-NeRF | FID: 183.25 -> 185.79 | Hurt |
| add LPIPS loss on top of adversarial training (ablation) | MALD-NeRF · SPIn-NeRF | LPIPS: 0.4345 -> 0.4130 | Mixed |
| add pixel loss on top of adversarial training (ablation) | MALD-NeRF · SPIn-NeRF | FID: 183.25 -> 256.82 | Hurt |
| add per-scene LoRA finetuning of the latent diffusion model (ablation) | MALD-NeRF · SPIn-NeRF | FID: 224.29 -> 183.25 | Improved |
| add discriminator feature matching loss (ablation) | MALD-NeRF · SPIn-NeRF | FID: 232.28 -> 183.25 | Improved |
| add masking of reconstruction/inpainting boundary in adversarial training (ablation) | MALD-NeRF · SPIn-NeRF | FID: 196.47 -> 183.25 | Improved |
| use MALD-NeRF over InpaintNeRF360 | NeRF inpainting · SPIn-NeRF | FID: 222.12 -> 183.25 | Improved |
| use latent diffusion model inpainter in SPIn-NeRF instead of LaMa | SPIn-NeRF · SPIn-NeRF | FID: 219.80 -> 227.87 | Hurt |
Closest papers by meaning
- DiffusioNeRF: Regularizing Neural Radiance Fields With Denoising Diffusion Models (CVPR 2023 · similarity 0.93)
- Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction (ICCV 2023 · similarity 0.92)
- DiffRF: Rendering-Guided 3D Radiance Field Diffusion (CVPR 2023 · similarity 0.92)
- Removing Objects From Neural Radiance Fields (CVPR 2023 · similarity 0.91)
- Mask-Based Modeling for Neural Radiance Fields (ICLR 2024 · similarity 0.90)
- Rad-NeRF: Ray-decoupled Training of Neural Radiance Field (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
