Research map / Video / Video segmentation and tracking
Latency Correction for Event-guided Deblurring and Frame Interpolation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a self-supervised, differentiable latency correction for event cameras (parameterized latency-illuminance polynomial plus event-based temporal fidelity loss) that improves event-guided deblurring and frame interpolation under low light.
Problem
Event camera latency, which varies with illuminance and motion, causes timestamp errors that degrade time-sensitive event-guided deblurring and frame interpolation, especially in low-light scenes.
Approach
Model latency as a K-th order polynomial of illuminance (approximated by the blurry image), reformulate the event-based double integral (EDI) to be differentiable with respect to latency, and optimize latency self-supervisedly using a new event-based temporal fidelity objective that measures sharpness of the reconstructed latent image.
Key results
- On synthetic HighREV deblurring with EDI, constant latency correction raised PSNR from 26.001 to 27.471 (GT-input 28.489), a +5.15% improvement relative to the gap.
- With GEM deblurring, PSNR improved from 27.714 (uncorrected) to 30.372 (corrected), versus 30.611 for ground-truth events.
- With REFID interpolation, PSNR improved from 28.244 to 34.217 (GT input 34.522), a 17.30% relative gap closure.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply constant latency correction | EDI · HighREV synthetic, deblurring | PSNR: 26.001 -> 27.471 | Helped |
| Apply constant latency correction | EDI · HighREV synthetic, deblurring | SSIM: 0.810 -> 0.828 | Helped |
| Apply constant latency correction | EDI · HighREV synthetic, deblurring | LPIPS: 0.500 -> 0.488 | Helped |
| Apply constant latency correction | EFNet · HighREV synthetic, deblurring | PSNR: 33.330 -> 33.778 | Helped |
| Apply constant latency correction | EFNet · HighREV synthetic, deblurring | SSIM: 0.971 -> 0.971 | No clear effect |
| Apply constant latency correction | EFNet · HighREV synthetic, deblurring | LPIPS: 0.107 -> 0.106 | Helped |
| Apply constant latency correction | GEM · HighREV synthetic, deblurring | PSNR: 27.714 -> 30.372 | Helped |
| Apply constant latency correction | GEM · HighREV synthetic, deblurring | SSIM: 0.934 -> 0.958 | Helped |
| Apply constant latency correction | GEM · HighREV synthetic, deblurring | LPIPS: 0.198 -> 0.185 | Helped |
| Apply constant latency correction | EDI · HighREV synthetic, interpolation | PSNR: 22.125 -> 22.265 | Helped |
| Apply constant latency correction | EDI · HighREV synthetic, interpolation | SSIM: 0.834 -> 0.838 | Helped |
| Apply constant latency correction | EDI · HighREV synthetic, interpolation | LPIPS: 0.331 -> 0.331 | No clear effect |
Surprising findings
- EFNet is nearly unaffected by latency (33.330 vs 33.761 with GT input), likely because it implicitly modeled latency in training data generation.
Limitations
- Implementation is tailored to DAVIS346Mono cameras, which require perfect image-event synchronization.
- Illuminance is hard to measure, so the latency-illuminance curve parameters cannot be directly applied to arbitrary scenes.
- Real-data evaluation is only qualitative since no ground-truth sharp images exist.
Experimental setup
Datasets: HighREV self-collected DAVIS346 real-scene videos
Benchmarks: HighREV
Metrics: PSNR SSIM LPIPS
Baselines: EDI EFNet GEM REFID constant latency correction
Builds on: Event-based Double Integral (EDI) EFNet REFID GEM
Closest papers by meaning
- Frequency-aware Event-based Video Deblurring for Real-World Motion Blur (CVPR 2024 · similarity 0.88)
- LED: A Large-scale Real-world Paired Dataset for Event Camera Denoising (CVPR 2024 · similarity 0.87)
- Mitigating Motion Blur in Neural Radiance Fields with Events and Frames (CVPR 2024 · similarity 0.86)
- Towards Robust Event-guided Low-Light Image Enhancement: A Large-Scale Real-World Event-Image Dataset and Novel Approach (CVPR 2024 · similarity 0.86)
- Seeing Motion at Nighttime with an Event Camera (CVPR 2024 · similarity 0.86)
- Motion Deblurring via Spatial-Temporal Collaboration of Frames and Events (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.