Research map / 3D vision / NeRF and radiance fields
OmniLocalRF: Omnidirectional Local Radiance Fields from Dynamic Videos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OmniLocalRF, a local-radiance-field method with bidirectional optimization over distant omnidirectional frames and a multi-resolution feature-plane motion mask, to render static scenes from casual dynamic 360° videos without masks or pose priors.
Problem
Synthesizing static-scene novel views from long casually captured 360° videos where dynamic objects, including the photographer, cause ghosting artifacts and occlusions.
Approach
Combines LocalRF's sliding-window radiance-field blocks with a bidirectional forward/backward optimization across distant frames enabled by omnidirectional coverage, plus a self-supervised multi-resolution neural feature-plane mask module that segments dynamic objects jointly with rendering.
Key results
- On the real dataset, OmniLocalRF without poses reaches 27.73 PSNR and 0.3297 LPIPS, versus 26.88 PSNR for Mip-NeRF360 and 26.56 for LocalRF with poses.
- On the synthetic dataset, OmniLocalRF without poses achieves 29.93 PSNR versus 25.59 for Mip-NeRF360 and 25.22 for LocalRF without poses.
- In pose estimation on synthetic videos, OmniLocalRF lowers ATE to 0.00165 compared with 0.00376 for LocalRF and 0.00218 for OpenMVG.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OmniLocalRF with camera pose priors instead of Mip-NeRF360 | OmniLocalRF · Synthetic dataset | PSNR: 25.59 -> 29.76 | Helped |
| Use OmniLocalRF without camera pose priors instead of Mip-NeRF360 | OmniLocalRF · Real dataset | PSNR: 26.88 -> 27.73 | Helped |
| Use OmniLocalRF instead of LocalRF for pose estimation | synthetic videos with dynamic objects · Synthetic dataset | ATE: 0.00376 -> 0.00165 | Helped |
| Remove the forward step (ablation) | OmniLocalRF · Real dataset | PSNR: 27.73 -> 27.64 | Hurt |
| Omit source-ray photometric supervision in the backward step (ablation) | OmniLocalRF · Real dataset | PSNR: 27.73 -> 26.23 | Hurt |
| Remove the backward step (ablation) | OmniLocalRF · Real dataset | PSNR: 27.73 -> 27.63 | Hurt |
| Remove the motion mask module (ablation) | OmniLocalRF · Real dataset | PSNR: 27.73 -> 26.08 | Hurt |
| Remove mask photometric supervision (ablation) | OmniLocalRF · Real dataset | PSNR: 27.73 -> 25.27 | Hurt |
Surprising findings
- D2NeRF and RobustNeRF, designed for dynamic-scene static reconstruction, score lower than general-purpose methods like Mip-NeRF360 on 360° videos due to limited capacity.
Limitations
- Cannot inpaint regions completely occluded in the input video because training relies on photometric loss.
- Equirectangular linear interpolation causes oversampling near polar regions in mask prediction.
- No global bundle adjustment or loop closure, so pose estimation lacks SLAM-level robustness.
Experimental setup
Datasets: self-captured Insta360 real 360° video dataset (6 outdoor scenes) synthetic 360° videos with floating dynamic objects (3 scenes)
Benchmarks: self-captured real 360° video dataset (6 scenes) synthetic 360° video dataset (3 scenes)
Metrics: PSNR SSIM LPIPS PSNR-WS SSIM-WS ATE RPE
Baselines: Mip-NeRF360 EgoNeRF LocalRF D2NeRF RobustNeRF OpenMVG
Builds on: LocalRF TensoRF DynIBaR Mip-NeRF 360
Closest papers by meaning
- SC-OmniGS: Self-Calibrating Omnidirectional Gaussian Splatting (ICLR 2025 · similarity 0.88)
- ODGS: 3D Scene Reconstruction from Omnidirectional Images with 3D Gaussian Splattings (NeurIPS 2024 · similarity 0.88)
- OmniSplat: Taming Feed-Forward 3D Gaussian Splatting for Omnidirectional Images with Editable Capabilities (CVPR 2025 · similarity 0.86)
- OmniSDF: Scene Reconstruction using Omnidirectional Signed Distance Functions and Adaptive Binoctrees (CVPR 2024 · similarity 0.86)
- Joint Optimization of Neural Radiance Fields and Continuous Camera Motion from a Monocular Video (CVPR 2025 · similarity 0.86)
- BLiRF: Bandlimited Radiance Fields for Dynamic Scene Modeling (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.