Research map / 3D vision / Gaussian splatting
MetaGS: A Meta-Learned Gaussian-Phong Model for Out-of-Distribution 3D Scene Relighting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
MetaGS couples a meta-learned (bilevel) 3D Gaussian splatting training scheme with a differentiable Blinn-Phong shading model to relight scenes under out-of-distribution point-light positions from OLAT captures.
Problem
Novel view relighting from one-light-at-a-time (OLAT) captures fails when test light positions are out of the training light distribution (spatially biased lights), causing overfitting and unrealistic specular highlights and shadows.
Approach
Treat each lighting condition as a task and apply bilevel (MAML-style) meta-learning to 3DGS so geometry and appearance generalize across lights, combined with learnable per-Gaussian normals, diffuse/specular coefficients and a Blinn-Phong shading model with ray-traced shadows.
Key results
- On OOD relighting, MetaGS averages 25.83 PSNR across 10 scenes versus 21.18 for GS$^3$ (best baseline average is NRHints at 22.64 PSNR per the full table).
- Ablation on synthetic scenes: full model 27.42 PSNR vs 19.14 without meta-learning and 21.53 without shadow computation.
- Camera-light-colocated synthetic setting: MetaGS reaches 38.72 PSNR on Layered Woven Ball versus 26.99 for IRON.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add meta-learning to MetaGS (ablation) | MetaGS · Three synthetic scenes (OOD average) | PSNR: 19.14 -> 27.42 | Helped |
| Add meta-learning to MetaGS (ablation) | MetaGS · Three synthetic scenes (OOD average) | SSIM: 0.8781 -> 0.9546 | Helped |
| Add meta-learning to MetaGS (ablation) | MetaGS · Three synthetic scenes (OOD average) | LPIPS: 0.0892 -> 0.0505 | Helped |
| Add shadow computation to MetaGS (ablation) | MetaGS · Three synthetic scenes (OOD average) | PSNR: 21.53 -> 27.42 | Helped |
| Add shadow computation to MetaGS (ablation) | MetaGS · Three synthetic scenes (OOD average) | SSIM: 0.9105 -> 0.9546 | Helped |
| Add shadow computation to MetaGS (ablation) | MetaGS · Three synthetic scenes (OOD average) | LPIPS: 0.0735 -> 0.0505 | Helped |
| Increase task batch size without meta-learning (ablation) | MetaGS training · Three synthetic scenes (OOD average) | PSNR: 19.14 -> 20.94 | Helped |
| Use meta-learning rather than only increasing batch size (ablation) | MetaGS training · Three synthetic scenes (OOD average) | PSNR: 20.94 -> 27.42 | Helped |
| Use MetaGS instead of NRHints | OOD relighting · Layered Woven Ball | PSNR: 17.25 -> 26.76 | Helped |
| Use MetaGS instead of NRHints | OOD relighting · Plastic Cup | PSNR: 23.92 -> 27.54 | Helped |
| Use MetaGS instead of NRHints | OOD relighting · Rubber Cup | PSNR: 27.44 -> 27.95 | Helped |
| Use MetaGS instead of IRON | Camera-light-colocated relighting · Layered Woven Ball | PSNR: 26.99 -> 38.72 | Helped |
Surprising findings
- Baseline NRHints scores higher than MetaGS on in-distribution Plastic Cup and Rubber Cup scenes, which the authors attribute to implicit color fitting.
Limitations
- Only direct illumination is modeled; indirect lighting is not accounted for.
- Simple Blinn-Phong reflection may limit fidelity for subsurface scattering or anisotropic materials.
- Robustness under more complex lighting conditions is an open challenge.
Experimental setup
Datasets: NRHints synthetic scenes (Layered Woven Ball, Plastic Cup, Rubber Cup) rendered in Blender NRHints real-world scenes (7 scenes)
Benchmarks: NRHints synthetic scenes NRHints real-world scenes
Metrics: PSNR SSIM LPIPS
Baselines: NRHints WildLight GS$^3$ IRON Relightable 3DGS GaussianShader R3G 3DGS
Builds on: 3D Gaussian splatting MAML Blinn-Phong reflection model GaussianShader NRHints
Closest papers by meaning
- RNG: Relightable Neural Gaussians (CVPR 2025 · similarity 0.91)
- ReCap: Better Gaussian Relighting with Cross-Environment Captures (CVPR 2025 · similarity 0.90)
- UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting (NeurIPS 2025 · similarity 0.89)
- GI-GS: Global Illumination Decomposition on Gaussian Splatting for Inverse Rendering (ICLR 2025 · similarity 0.89)
- Neural Gaffer: Relighting Any Object via Diffusion (NeurIPS 2024 · similarity 0.89)
- UniLumos: Fast and Unified Image and Video Relighting with Physics-Plausible Feedback (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.