Research map / 3D vision / Gaussian splatting
RelitLRM: Generative Relightable Radiance for Large Reconstruction Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RelitLRM, a feed-forward transformer that reconstructs relightable 3D Gaussian splats from 4-8 posed images under unknown lighting, pairing a deterministic GS-LRM-style geometry reconstructor with a diffusion-based relit appearance generator.
Problem
Relightable 3D object reconstruction from sparse posed images under unknown static lighting normally needs dense views and slow per-scene optimization, and regression-based feed-forward models miss multi-modal specular highlights and shadows.
Approach
Decouple geometry and appearance: a deterministic transformer predicts 3DGS geometry, then a second transformer denoiser (relit-view diffusion with x0 prediction) conditioned on target environment-map tokens and geometry tokens generates per-Gaussian SH color, trained end-to-end from scratch on large synthetic relighting data.
Key results
- On Stanford-ORB with 6 input views, RelitLRM reaches PSNR-H 24.67 and PSNR-L 31.52 in about 2 seconds, versus 26.01/33.26 for Neural-PBIR using 42-67 views over hours.
- On TensoIR-Synthetic with 8 views, average PSNR is 29.05 and LPIPS 0.082 in about 3 seconds, comparable to TensoIR (28.58 PSNR, 100 views, hours).
- On the held-out Objaverse set, RelitLRM achieves PSNR 28.50 versus 24.51 for NeuralGaffer, with 5 sampling steps and 0.4B parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use probabilistic diffusion relighting rather than deterministic relighting (ablation) | RelitLRM · Held-out validation set | LPIPS: 0.076 -> 0.064 | Helped |
| Add probabilistic diffusion relighting rather than deterministic relighting (ablation) | RelitLRM · Held-out validation set | PSNR: 27.10 -> 27.63 | Helped |
| Increase input views from 2 to 8 (ablation) | RelitLRM Res-512 · Held-out validation set | PSNR: 23.58 -> 28.08 | Helped |
| Increase input views from 4 to 16 (ablation) | RelitLRM Res-512 · TensoIR-Synthetic | PSNR: 27.83 -> 29.31 | Helped |
| Increase denoising views from 1 to 4 (ablation) | RelitLRM · Held-out validation set | PSNR: 27.05 -> 27.63 | Helped |
| Increase classifier-free guidance from 1.0 to 3.0 (ablation) | RelitLRM · Held-out validation set | PSNR: 27.00 -> 27.63 | Helped |
| Increase classifier-free guidance from 3.0 to 6.0 (ablation) | RelitLRM · Held-out validation set | PSNR: 27.63 -> 23.64 | Hurt |
| Increase denoising steps from 2 to 5 (ablation) | RelitLRM · Held-out validation set | PSNR: 23.81 -> 27.63 | Helped |
| Increase denoising steps from 5 to 10 (ablation) | RelitLRM · Held-out validation set | PSNR: 27.63 -> 27.00 | Hurt |
| Use 4 denoising views instead of 2 (ablation) | RelitLRM · Held-out validation set | PSNR: 26.87 -> 27.63 | Helped |
| Use RelitLRM rather than NeuralGaffer | RelitLRM · Held-out validation set | PSNR: 24.51 -> 28.50 | Helped |
| Use RelitLRM rather than Neural-PBIR | RelitLRM · Stanford-ORB | PSNR-H: 26.01 -> 24.67 | Hurt |
Surprising findings
- The deterministic ablation achieves similar aggregate metrics to the probabilistic version, with the main gain being visual quality on specular objects.
- Performance saturates around 16 input views, and CFG of 6.0 degrades results sharply (PSNR 23.64) relative to 3.0.
Limitations
- Requires known camera parameters for input views.
- Performance saturates beyond about 16 input views, and resolution scalability remains open.
- Environment-map lighting cannot represent near-field lighting accurately.
Experimental setup
Datasets: Objaverse Zeroverse Polyhaven HDRIs Laval Indoor Laval Outdoor
Benchmarks: Stanford-ORB Objects-with-Lighting TensoIR-Synthetic held-out Objaverse set
Metrics: PSNR SSIM LPIPS runtime
Baselines: NVDiffRec-MC InvRender Neural-PBIR TensoIR NeRFactor FlashCache NeuralGaffer DiLightNet
Builds on: GS-LRM 3D Gaussian Splatting DDIM classifier-free guidance
Closest papers by meaning
- Generative Multiview Relighting for 3D Reconstruction under Extreme Illumination Variation (CVPR 2025 · similarity 0.91)
- pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction (CVPR 2024 · similarity 0.89)
- IllumiNeRF: 3D Relighting Without Inverse Rendering (NeurIPS 2024 · similarity 0.89)
- SVG-IR: Spatially-Varying Gaussian Splatting for Inverse Rendering (CVPR 2025 · similarity 0.88)
- LIRM: Large Inverse Rendering Model for Progressive Reconstruction of Shape, Materials and View-dependent Radiance Fields (CVPR 2025 · similarity 0.88)
- UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.