Research map / 3D vision / Gaussian splatting
Unifying Appearance Codes and Bilateral Grids for Driving Scene Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a multi-scale bilateral grid pyramid that unifies appearance codes and bilateral grids to improve photometric consistency and geometry in dynamic driving-scene Gaussian Splatting reconstruction.
Problem
Photometric inconsistency across cameras and times in driving scenes degrades neural rendering geometry; appearance codes are too global and single bilateral grids are hard to optimize.
Approach
A three-level pyramid of bilateral grids (coarse 2x2x1, intermediate 4x4x2, fine 8x8x4, 12 affine coefficients each) with luminance-guided slicing at downsampled guidance resolutions, composed coarse-to-fine; it reduces to appearance codes at the coarsest scale and pixel-wise bilateral grids at the finest.
Key results
- On NuScenes the method reaches 27.69 reconstruction PSNR and 1.161 CD versus OmniRe's 26.37 PSNR and 1.458 CD.
- On Waymo CD drops to 0.989 from 1.378 for the best baseline (OmniRe w/AC), a 28.2% reduction.
- Applied to StreetGS on NuScenes, reconstruction PSNR rises from 25.74 to 27.90 and CD from 1.604 to 1.272.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the multi-scale bilateral grid instead of OmniRe | OmniRe on Waymo · Waymo | CD: 1.482 -> 0.989 | Helped |
| Use the multi-scale bilateral grid instead of OmniRe | OmniRe on NuScenes · NuScenes | CD: 1.458 -> 1.161 | Helped |
| Use the multi-scale bilateral grid instead of OmniRe | OmniRe on Argoverse · Argoverse | CD: 0.954 -> 0.807 | Helped |
| Use the multi-scale bilateral grid instead of OmniRe | OmniRe on PandaSet · PandaSet | CD: 0.503 -> 0.453 | Helped |
| Add multi-scale bilateral grids | ChatSim on NuScenes · NuScenes | CD: 1.557 -> 1.236 | Helped |
| Add multi-scale bilateral grids | StreetGS on NuScenes · NuScenes | CD: 1.604 -> 1.272 | Helped |
| Use guidance downsampling factors (1,1,1) instead of (2,2,1) (ablation) | Multi-scale bilateral grid · NuScenes | NVS PSNR: 24.64 -> 24.34 | Hurt |
| Use guidance downsampling factors (8,8,4) instead of (2,2,1) (ablation) | Multi-scale bilateral grid · NuScenes | NVS PSNR: 24.64 -> 24.53 | Hurt |
| Use guidance downsampling factors (16,16,8) instead of (2,2,1) (ablation) | Multi-scale bilateral grid · NuScenes | NVS PSNR: 24.64 -> 24.54 | Hurt |
| Remove the coarse and intermediate levels, retaining only a single fine grid (ablation) | Multi-scale bilateral grid · NuScenes | CD: 1.161 -> 1.376 | Hurt |
| Remove the coarsest grid level (2,2,1) (ablation) | Multi-scale bilateral grid · NuScenes | CD: 1.161 -> 1.963 | Hurt |
| Remove the intermediate grid level (4,4,2) (ablation) | Multi-scale bilateral grid · NuScenes | CD: 1.161 -> 1.940 | Hurt |
Surprising findings
- A single bilateral grid (OmniRe w/BG) often fails to converge and can degrade performance relative to appearance codes.
- The multi-scale grid uses about 3.97M parameters versus 27.8M for a single large bilateral grid, with only a 9% training-time increase over appearance codes.
Limitations
- Computational overhead is higher than simple appearance codes.
- Fast-moving or highly non-rigid objects with LiDAR-camera misalignment remain challenging.
Experimental setup
Datasets: Waymo NuScenes Argoverse2 PandaSet
Benchmarks: Waymo NuScenes Argoverse2 PandaSet
Metrics: PSNR SSIM LPIPS Chamfer Distance RMSE Median depth error
Baselines: OmniRe OmniRe w/AC OmniRe w/BG ChatSim StreetGS DriveStudio
Builds on: bilateral grid OmniRe deep bilateral learning appearance codes
Closest papers by meaning
- EGSRAL:An Enhanced 3D Gaussian Splatting Based Renderer with Automated Labeling for Large-Scale Driving Scene (AAAI 2025 · similarity 0.89)
- SplatAD: Real-Time Lidar and Camera Rendering with 3D Gaussian Splatting for Autonomous Driving (CVPR 2025 · similarity 0.89)
- Novel View Synthesis Under Large-Deviation Viewpoint for Autonomous Driving (AAAI 2025 · similarity 0.88)
- DrivingGaussian: Composite Gaussian Splatting for Surrounding Dynamic Autonomous Driving Scenes (CVPR 2024 · similarity 0.88)
- GGS: Generalizable Gaussian Splatting for Lane Switching in Autonomous Driving (AAAI 2025 · similarity 0.88)
- DrivingForward: Feed-forward 3D Gaussian Splatting for Driving Scene Reconstruction from Flexible Surround-view Input (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.