Research map / 3D vision / 3D generation
S2D-LFE: Sparse-to-Dense Light Field Event Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes S2D-LFE, a diffusion-based network with an LFE-customized VAE and LFE-adapter that generates dense light field event views from sparse 2x2 event inputs, plus a real capture system and synthetic Carla dataset.
Problem
Generating dense, free-viewpoint, temporally consistent light field event (LFE) data from sparse-view LFE input without relying on frame-based cameras.
Approach
A diffusion pipeline with an LFE-VAE (angular interaction blocks and sparse-view replacement in the decoder) and an LFE-adapter that injects geometric (EPI-based) and texture priors from sparse LFE views as multi-scale conditioning, using continuous angular coordinate prompts for arbitrary view positions.
Key results
- On 5x5 in-scale synthetic LFE, S2D-LFE reaches 24.06 dB PSNR and 0.701 SSIM, 0.60 dB above the second-best method (Guo et al., 23.46 dB).
- On out-of-scale 8x8 (64-view) generation, S2D-LFE keeps better NIQE/BRISQUE/MUSIQ than Guo et al. and R2L across synthetic and real sets.
- LFE-VAE raises PSNR to 38.54 dB versus 36.89 dB for the standard VAE in the 25-view ablation; LFE-adapter improves PSNR from 22.54 to 24.06 dB over concatenation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use S2D-LFE instead of Guo et al. | LFE view synthesis · LFE-syn, 5×5 | PSNR: 23.46 -> 24.06 | Helped |
| use S2D-LFE instead of Guo et al. | LFE view synthesis · LFE-syn, 5×5 | SSIM: 0.687 -> 0.701 | Helped |
| use S2D-LFE instead of Guo et al. | LFE view synthesis · LFE-syn, 5×5 | LPIPS: 0.290 -> 0.239 | Helped |
| use S2D-LFE instead of Guo et al. | LFE view synthesis · LFE-real, 5×5 | NIQE: 28.23 -> 27.65 | Helped |
| use S2D-LFE instead of Guo et al. | LFE view synthesis · LFE-real, 5×5 | BRISQUE: 59.13 -> 53.45 | Helped |
| use S2D-LFE instead of Guo et al. | LFE view synthesis · LFE-real, 5×5 | MUSIQ: 30.72 -> 31.14 | Helped |
| add LFE-VAE instead of using a standard VAE (ablation) | S2D-LFE · LFE-syn, 5×5 reconstruction | PSNR: 36.89 -> 38.54 | Helped |
| add LFE-VAE instead of fine-tuning a VAE on single-view event data (ablation) | S2D-LFE · LFE-syn, 5×5 reconstruction | PSNR: 37.12 -> 38.54 | Helped |
| use the LFE-adapter instead of simple feature concatenation (ablation) | S2D-LFE · LFE-syn, 5×5 generation | PSNR: 22.54 -> 24.06 | Helped |
Limitations
- Direct comparison with Ev-LFV is not feasible because it uses different input modalities (RGB frames plus events).
- Real-world capture uses a small 2x2 rig with fixed 6 cm baselines and only 25 sequences.
- Only 5x5 training scale is evaluated in-distribution; out-of-scale results depend on generalization of angular coordinate prompts.
Experimental setup
Datasets: LFE-syn (Carla synthetic, 162 sequences) LFE-real (25 sequences captured with 2x2 DAVIS346 rig)
Benchmarks: LFE-syn LFE-real
Metrics: PSNR SSIM LPIPS NIQE BRISQUE MUSIQ
Baselines: Jing et al. DistgASR SAV Guo et al. ET-Net+SAV+Vid2E ET-Net+Guo et al.+Vid2E R2L
Builds on: Stable Diffusion VAE (SDXL) ControlNet-style adapter (T2I-Adapter) Ev-LFV DistgASR
Closest papers by meaning
- From Pixels to Views: Learning Angular-Aware and Physics-Consistent Representations for Light Field Microscopy (NeurIPS 2025 · similarity 0.85)
- Towards Robust Event-guided Low-Light Image Enhancement: A Large-Scale Real-World Event-Image Dataset and Novel Approach (CVPR 2024 · similarity 0.85)
- NeLF-Pro: Neural Light Field Probes for Multi-Scale Novel View Synthesis (CVPR 2024 · similarity 0.85)
- Event Fields: Capturing Light Fields at High Speed, Resolution, and Dynamic Range (CVPR 2025 · similarity 0.84)
- Mitigating Motion Blur in Neural Radiance Fields with Events and Frames (CVPR 2024 · similarity 0.84)
- Novel View Synthesis from A Few Glimpses via Test-Time Natural Video Completion (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.