Research map / Video / Video segmentation and tracking
Unified Reconstruction of Static and Dynamic Scenes from Events
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes URSEE, a two-stage event-camera framework that reconstructs static backgrounds via convolutional integration plus a denoising network, and fuses them with dynamic event voxel grids in an end-to-end recurrent U-Net to reconstruct videos with static and dynamic content.
Problem
Event-based video reconstruction methods fail to recover static background information, since static scenes trigger few events and existing integration-based static reconstruction yields noisy, saturated images; no prior method reconstructs both static backgrounds and moving foregrounds end-to-end.
Approach
Replace pixel-wise event integration with 3x3 convolutional (mean-filter) integration to avoid saturation, denoise with a U-Net-based SRD module, separate static and dynamic events via spatiotemporal windows, and feed the static background plus dynamic voxel grids with an event-separation label tensor into a ConvLSTM U-Net (ERSD) for joint video reconstruction.
Key results
- On the E-Static dataset, the proposed static reconstruction achieves PSNR 22.43 and SSIM 0.860, versus 17.94 PSNR for retrained E2VID.
- On the E-StaDyn synthetic dataset, URSEE reaches PSNR 31.97, SSIM 0.959, LPIPS 0.122, versus 18.45 PSNR for retrained E2VID.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use URSEE instead of E2VID (retrained) | Static image reconstruction · E-Static | PSNR: 17.94 -> 22.43 | Helped |
| Use URSEE instead of E2VID (retrained) | Static image reconstruction · E-Static | SSIM: 0.658 -> 0.860 | Helped |
| Use URSEE instead of E2VID (retrained) | Static image reconstruction · E-Static | LPIPS: 0.369 -> 0.244 | Helped |
| Use URSEE instead of E2VID (retrained) | Dynamic video reconstruction · E-StaDyn | PSNR: 18.45 -> 31.97 | Helped |
| Use URSEE instead of E2VID (retrained) | Dynamic video reconstruction · E-StaDyn | SSIM: 0.792 -> 0.959 | Helped |
| Use URSEE instead of E2VID (retrained) | Dynamic video reconstruction · E-StaDyn | LPIPS: 0.448 -> 0.122 | Helped |
| Add convolutional integration instead of pixel-wise integration (ablation) | Static scene reconstruction · 96 test scenes | PSNR: 5.755 -> 11.471 | Helped |
| Add convolutional integration instead of pixel-wise integration (ablation) | Static scene reconstruction · 96 test scenes | SSIM: 0.022 -> 0.233 | Helped |
| Add convolutional integration instead of pixel-wise integration (ablation) | Static scene reconstruction · 96 test scenes | LPIPS: 1.381 -> 0.748 | Helped |
Surprising findings
- Pretrained E2VID partially recovers backgrounds only because moving foregrounds trigger background-related events, and retraining E2VID on the static-dynamic data still fails to reconstruct backgrounds reliably.
Limitations
- Static reconstruction quality hits a bottleneck under bright illumination due to event sensor design.
- Results are tied to the Prophesee EVK4 sensor; other devices need relearned response and noise models.
- Dataset coverage is limited, requiring further expansion for generalization.
Experimental setup
Datasets: E-Static E-StaDyn
Benchmarks: E-Static E-StaDyn
Metrics: PSNR SSIM LPIPS
Baselines: E2VID FireNet SSL ET-Net EVSNN E2HQV pixel-wise integration
Builds on: E2VID pixel-wise integration U-Net ConvLSTM
Closest papers by meaning
- LaSe-E2V: Towards Language-guided Semantic-aware Event-to-Video Reconstruction (NeurIPS 2024 · similarity 0.86)
- Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.84)
- EF-3DGS: Event-Aided Free-Trajectory 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.84)
- Event-3DGS: Event-based 3D Reconstruction Using 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.84)
- EvGGS: A Collaborative Learning Framework for Event-based Generalizable Gaussian Splatting (ICML 2024 · similarity 0.83)
- World from Motion: Generative Dynamic Gaussian Reconstruction from Monocular Video (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.