Research map / 3D vision / Gaussian splatting
Omni-Scene: Omni-Gaussian Representation for Ego-Centric Sparse-View Scene Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Omni-Scene, which combines voxel-anchored volume-based Gaussians and pixel-aligned Gaussians (Omni-Gaussian) for feed-forward ego-centric sparse-view reconstruction from surround-view driving cameras.
Problem
Feed-forward sparse-view 3D Gaussian reconstruction relies on cross-view overlap and full visibility, so pixel-based Gaussians fail for ego-centric driving captures with minimal overlap, occlusions and frustum truncations.
Approach
Combine a volume-based Gaussian branch (triplane transformer with cross-image and cross-plane deformable attention, voxel-anchored Gaussians) that handles occlusion and truncation with a pixel-based branch that adds fine details and distant Gaussians, linked by projection-based feature fusion and depth-guided training decomposition.
Key results
- On ego-centric nuScenes, Omni-Scene reaches 24.01 PSNR, 0.733 SSIM, 0.238 LPIPS and 0.804 PCC, versus 21.61 PSNR for MVSplat and 21.51 for pixelSplat.
- On RealEstate10K, Omni-Scene achieves 26.19 PSNR and 0.368 PCC, comparable to MVSplat's 26.39 PSNR, while outperforming all listed methods on PCC.
- Ablation: removing the volume branch drops PSNR by 1.12 dB, and removing the pixel branch drops PSNR by 1.8 dB on nuScenes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add volume-based Gaussians to the pixel-based representation (ablation) | Omni-Scene · nuScenes | PSNR: 22.89 -> 24.01 | Helped |
| add pixel-based Gaussians to the volume-based representation (ablation) | Omni-Scene · nuScenes | PSNR: 22.21 -> 24.01 | Helped |
| add Projection-Based Feature Fusion (ablation) | Omni-Scene · nuScenes | PSNR: 23.35 -> 24.01 | Helped |
| add Depth-Guided Training Decomposition (ablation) | Omni-Scene · nuScenes | PSNR: 23.75 -> 24.01 | Helped |
| initialize pixel depths using the 2D foundation model (ablation) | pixel-based Gaussian variant · nuScenes | PCC: 0.572 -> 0.780 | Helped |
Limitations
- Volume-based Gaussians are bounded to a fixed range around the vehicle, so distant elements require the pixel branch.
- Volume resolution is limited by cubic complexity, which restricts detail.
Experimental setup
Datasets: nuScenes RealEstate10K
Benchmarks: nuScenes RealEstate10K
Metrics: PSNR SSIM LPIPS PCC
Baselines: pixelSplat MVSplat AttnRend MuRF
Builds on: pixelSplat MVSplat Depth Anything
Closest papers by meaning
- ODGS: 3D Scene Reconstruction from Omnidirectional Images with 3D Gaussian Splattings (NeurIPS 2024 · similarity 0.87)
- FRUC: Feedforward Dynamic Scene Reconstruction from Uncalibrated Collaborative Driving Views (NeurIPS 2026 · similarity 0.86)
- DrivingForward: Feed-forward 3D Gaussian Splatting for Driving Scene Reconstruction from Flexible Surround-view Input (AAAI 2025 · similarity 0.86)
- No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images (ICLR 2025 · similarity 0.86)
- ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videos (CVPR 2025 · similarity 0.86)
- OmniRe: Omni Urban Scene Reconstruction (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.