Research map / 3D vision / Depth and stereo
Time-Efficient Light-Field Acquisition Using Coded Aperture and Events
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Combines a coded aperture (LCoS-displayed pattern sequence) with an event camera so that four coding patterns are applied within one exposure, and jointly trains an end-to-end acquisition and light-field reconstruction network for single-exposure light-field capture.
Problem
Light-field acquisition with coded-aperture imaging needs several sequential exposures, making measurement slow; joint aperture-exposure coding is hard to implement in hardware.
Approach
Apply a sequence of aperture coding patterns during a single exposure and record the parallax-induced intensity changes as events from an event camera; the frame plus event stacks are shown to be quasi-equivalent to multi-shot coded-aperture images, and a differentiable event-simulation pipeline is trained end-to-end with a CNN (RecNet).
Key results
- Flexible-tau model reaches 33.79 dB PSNR / 0.9149 SSIM averaged over BasicLFSR at test tau=0.15, outperforming single-shot baselines such as JAEC (N=4) + RecNet (31.97 dB) and LA + RecNet (26.22 dB).
- Fixed-tau-low model (33.56 dB average trained/tested at 0.075) is close to the multi-shot CA (N=4) upper bound at 35.39 dB.
- Prototype with DAVIS 346 and LCoS display completes acquisition in about 22 msec and produces visually convincing light fields of real 3-D scenes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the joint image-frame and event-stack measurement instead of image-only acquisition (ablation) | Light-field reconstruction · BasicLFSR test (ALL) | PSNR: 26.85 -> 33.79 | Helped |
| Use the joint image-frame and event-stack measurement instead of event-only acquisition (ablation) | Light-field reconstruction · BasicLFSR test (ALL) | PSNR: 28.35 -> 33.79 | Helped |
| Fix τ to 0.15 during training instead of sampling it from [0.075, 0.3] (ablation) | Flexible-τ · BasicLFSR test (ALL), test τ = 0.15 | PSNR: 33.79 vs. 33.83 | No clear effect |
| Fix τ to 0.3 during training instead of sampling it from [0.075, 0.3] (ablation) | Flexible-τ · BasicLFSR test (ALL), test τ = 0.3 | PSNR: 31.85 vs. 33.39 | Hurt |
| Train with randomly sampled τ instead of fixing τ to 0.075 (ablation) | Light-field reconstruction · BasicLFSR test (ALL), test τ = 0.075 | PSNR: 34.45 vs. 33.56 | Hurt |
| Enable the second loss term for the event-only model (ablation) | Events-only light-field reconstruction · BasicLFSR test (ALL), test τ = 0.15 | PSNR: 28.35 -> 14.81 | Hurt |
| Use the proposed single-exposure method rather than CA (N = 1) | Flexible-τ light-field reconstruction · BasicLFSR test (ALL), test τ = 0.15 | PSNR: 27.62 -> 33.79 | Helped |
| Use the proposed single-exposure method rather than Full-4D + RecNet | Flexible-τ light-field reconstruction · BasicLFSR test (ALL), test τ = 0.15 | PSNR: 33.22 -> 33.79 | Helped |
| Use the proposed single-exposure method rather than JAEC (N = 4) + RecNet | Flexible-τ light-field reconstruction · BasicLFSR test (ALL), test τ = 0.15 | PSNR: 31.97 -> 33.79 | Helped |
| Use flexible-τ training rather than CA (N = 2) + RecNet | Light-field reconstruction · BasicLFSR test (ALL), test τ = 0.15 | PSNR: 34.09 vs. 33.79 | No clear effect |
| Use the proposed method rather than CA (N = 4) + RecNet | Fixed-τ-low light-field reconstruction · BasicLFSR test (ALL) | PSNR: 35.39 vs. 34.45 | Hurt |
| Use flexible-τ training rather than fixed-τ training (ablation) | Light-field reconstruction · BasicLFSR test (ALL), across test-time τ values | PSNR | Mixed |
Surprising findings
- Event-only reconstruction with a learned pattern set achieves 28.35 dB, and the image-only ablation drops to 26.85 dB, showing that events carry much of the parallax information.
- A fixed-tau model is sensitive to test-time tau, whereas the flexible-tau model trained with random tau is robust over a wide range.
Limitations
- Relies on an event-camera contrast threshold that cannot be set directly; experiments are performed at tau around 0.15 with an estimated value.
- Only monochrome, 8x8 views, and static scenes are handled; RGB and higher resolution are future work.
- Quasi-equivalence to coded-aperture imaging holds only under an idealized equality assumption that real noisy quantized events break.
Experimental setup
Datasets: BasicLFSR
Benchmarks: BasicLFSR test set (EPFL, HCI new, HCI old, INRIA, Stanford)
Metrics: PSNR SSIM
Baselines: CA (N=4) + RecNet CA (N=2) + RecNet CA (N=1) + RecNet Full-4D + RecNet JAEC (N=4) + RecNet JAEC (N=4) LA + RecNet LA (naive)
Builds on: coded-aperture imaging joint aperture-exposure coding deep optics ESIM event simulator
Closest papers by meaning
- Coded-E2LF: Coded Aperture Light Field Imaging from Events (CVPR 2026 · similarity 0.93)
- Event Fields: Capturing Light Fields at High Speed, Resolution, and Dynamic Range (CVPR 2025 · similarity 0.90)
- Snapshot Lidar: Fourier Embedding of Amplitude and Phase for Single-Image Depth Reconstruction (CVPR 2024 · similarity 0.86)
- S2D-LFE: Sparse-to-Dense Light Field Event Generation (CVPR 2025 · similarity 0.84)
- Generalized Event Cameras (CVPR 2024 · similarity 0.84)
- Gated Fields: Learning Scene Reconstruction from Gated Videos (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.