Research map / 3D vision / NeRF and radiance fields
USB-NeRF: Unrolling Shutter Bundle Adjusted Neural Radiance Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes USB-NeRF, a single-stage NeRF that models rolling shutter image formation with a cubic B-spline camera motion model, jointly optimizing poses and scene to correct RS distortion and improve pose estimation.
Problem
NeRF assumes global shutter images, so rolling shutter distortion corrupts novel view synthesis and camera pose estimation; existing RS correction is two-stage and fragile.
Approach
Integrates the rolling shutter image formation model into NeRF so that the underlying 3D representation and per-scanline camera poses (modeled as cubic B-spline trajectories) are learned directly from RS images, enabling dense bundle adjustment and GS image rendering in one step.
Key results
- On synthetic RS sequences, USB-NeRF reaches 32.44 PSNR average versus 19.68 for BARF (rebuttal table).
- Rotation RPE on TUM-RS: USB-NeRF-cubic 0.227 deg/frame versus 0.429 for BARF and 2.095 for RSBA.
- On LLFF global shutter data, USB-NeRF averages 26.13 PSNR, comparable to NeRF (26.40) and BARF (25.91).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Model rolling-shutter image formation directly in NeRF instead of correcting images in a separate step | USB-NeRF · Synthetic rolling-shutter removal (Avg.) | PSNR: 19.68 -> 32.44 | Helped |
| Model rolling-shutter image formation directly in NeRF instead of correcting images in a separate step | USB-NeRF · Synthetic rolling-shutter removal (Avg.) | SSIM: 0.614 -> 0.914 | Helped |
| Model rolling-shutter image formation directly in NeRF instead of correcting images in a separate step | USB-NeRF · Synthetic rolling-shutter removal (Avg.) | LPIPS: 0.3364 -> 0.0521 | Helped |
| Use USB-NeRF on global-shutter images rather than NeRF | LLFF (Avg.) · LLFF | PSNR: 26.40 -> 26.13 | Mixed |
| Use USB-NeRF on global-shutter images rather than NeRF | LLFF (Avg.) · LLFF | SSIM: 0.855 -> 0.827 | Hurt |
| Use USB-NeRF on global-shutter images rather than NeRF | LLFF (Avg.) · LLFF | LPIPS: 0.0614 -> 0.0846 | Hurt |
| Use cubic B-Spline motion interpolation instead of linear interpolation (ablation) | USB-NeRF · Synthetic | RPE rotation (°/frame): 0.446$\pm$ 0.337 -> 0.307$\pm$ 0.140 | Helped |
| Use cubic B-Spline motion interpolation instead of linear interpolation (ablation) | USB-NeRF · TUM-RS | RPE rotation (°/frame): 0.283 $\pm$ 0.285 -> 0.227 $\pm$ 0.246 | Helped |
| Use cubic B-Spline motion interpolation instead of linear interpolation (ablation) | USB-NeRF · constant-velocity camera motion | PSNR: 0.25 dB worse | Hurt |
| Split BARF images into blocks and optimize each block pose separately | BARF · Synthetic rolling-shutter removal | PSNR | Mixed |
Surprising findings
- Splitting images into row blocks for BARF does not consistently improve results, sometimes hurting performance.
Limitations
- Assumes static scenes; dynamic environments not handled.
- Requires COLMAP initial poses.
- Cubic B-spline motion model may not fit highly nonlinear camera motion.
Experimental setup
Datasets: TUM-RS LLFF ETH3D-based synthetic RS sequences self-captured RS sequences (GoPro HERO6, Canon EOS M3, iPhone 14 Pro)
Benchmarks: TUM-RS LLFF synthetic RS sequences (Blue Room, Living Room, White Room)
Metrics: PSNR SSIM LPIPS ATE RPE
Baselines: NeRF BARF RSBA NW-RSBA COLMAP DSUN SUNet RSSR CVR DiffSfM
Builds on: NeRF BARF
Closest papers by meaning
- LuSh-NeRF: Lighting up and Sharpening NeRFs for Low-light Scenes (NeurIPS 2024 · similarity 0.88)
- UC-NERF: Neural Radiance Field for Under-Calibrated Multi-View Cameras in Autonomous Driving (ICLR 2024 · similarity 0.88)
- NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild (CVPR 2024 · similarity 0.87)
- CF-NeRF: Camera Parameter Free Neural Radiance Fields with Incremental Learning (AAAI 2024 · similarity 0.87)
- DynaMoDe-NeRF: Motion-aware Deblurring Neural Radiance Field for Dynamic Scenes (CVPR 2025 · similarity 0.86)
- Joint Optimization of Neural Radiance Fields and Continuous Camera Motion from a Monocular Video (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.