Research map / 3D vision / NeRF and radiance fields
AE-NeRF: Augmenting Event-Based Neural Radiance Fields for Non-ideal Conditions and Larger Scenes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AE-NeRF, a joint pose-correction and hierarchical event-distillation e-NeRF that reconstructs large scenes from event streams under noisy poses and non-uniform motion.
Problem
Event-based NeRF reconstruction fails under non-ideal conditions: noisy COLMAP poses from low-quality event-camera frames, non-uniform camera motion causing uneven event density, and scaling to larger unbounded scenes.
Approach
Jointly learns a time-conditioned pose correction network (time to SE(3)) with an e-NeRF using a proposal/vanilla two-network hierarchical event distillation, plus event-specific losses (reconstruction, temporal gradient, distortion) and a learned color correction network.
Key results
- On the proposed large-scene synthetic benchmark (hard setting), Ours (w. psi) reaches 24.23 PSNR versus 21.41 for Robust e-NeRF.
- On TUM-VIE real scenes, Ours (w. psi) achieves 19.69 PSNR and 0.1882 LPIPS, versus 18.97 PSNR for Robust e-NeRF.
- Pose correction reduces rotation error from 0.881 to 0.127 on the synthetic hard setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add pose correction network ψ (ablation) | AE-NeRF · Proposed synthetic scenes, easy and hard | PSNR: 22.83 -> 24.28; 22.79 -> 24.23 | Helped |
| add pose correction network ψ (ablation) | AE-NeRF · Synthetic hard pose-correction ablation | RE: 0.881 -> 0.127 | Helped |
| add pose correction network ψ (ablation) | AE-NeRF · TUM-VIE pose-correction ablation | RE: 1.283 -> 0.431 | Helped |
| add event temporal loss ℓg (ablation) | AE-NeRF with ℓp and ℓr · Loss ablation scenes | PSNR: 19.80 -> 23.19; +3.39dB | Helped |
| add event distortion loss ℓd (ablation) | AE-NeRF with ℓp and ℓr · Loss ablation scenes | PSNR: 19.80 -> 20.37; +0.57dB | Helped |
| combine event temporal loss ℓg and event distortion loss ℓd (ablation) | AE-NeRF with ℓp and ℓr · Loss ablation scenes | PSNR: 19.80 -> 24.67; +4.87dB | Helped |
| use AE-NeRF instead of Robust e-NeRF | Event-based novel view synthesis · TUM-VIE | PSNR: 18.97 -> 19.69 | Helped |
| use AE-NeRF instead of PAEv3D | Event-based novel view synthesis · PAEv3D dataset | PSNR: 25.903 -> 26.306; +0.403 dB | Helped |
Surprising findings
- Ablation shows the pose correction network gives only a small gain on object-level scenes but large gains on large scenes with non-uniform motion.
Limitations
- Experiments rely on synthetic event sequences because no real-world dataset with accurate poses and non-uniform motion is available.
- Real-world evaluation uses a subset of TUM-VIE, with only quantitative NVS on four scenes.
Experimental setup
Datasets: NeRF synthetic dataset Deblur-NeRF scenes TUM-VIE PAEv3D dataset
Benchmarks: NeRF synthetic (object-level) AE-NeRF large-scene synthetic dataset TUM-VIE PAEv3D dataset
Metrics: PSNR SSIM LPIPS rotation error translation error
Baselines: E2VID+NeRF Ev-NeRF Event NeRF Robust e-NeRF PAEv3D EventNeRF
Builds on: NeRF Mip-NeRF 360 Robust e-NeRF BARF E2VID
Closest papers by meaning
- Mitigating Motion Blur in Neural Radiance Fields with Events and Frames (CVPR 2024 · similarity 0.90)
- Event-3DGS: Event-based 3D Reconstruction Using 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.89)
- EventSplat: 3D Gaussian Splatting from Moving Event Cameras for Real-time Rendering (CVPR 2025 · similarity 0.88)
- DynaMoDe-NeRF: Motion-aware Deblurring Neural Radiance Field for Dynamic Scenes (CVPR 2025 · similarity 0.87)
- Language-driven Object Fusion into Neural Radiance Fields with Pose-Conditioned Dataset Updates (CVPR 2024 · similarity 0.87)
- Sync-NeRF: Generalizing Dynamic NeRFs to Unsynchronized Videos (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.