Research map / Point clouds / Point cloud classification
CodedEvents: Optimal Point-Spread-Function Engineering for 3D-Tracking with Event Cameras
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives Cramér-Rao bounds for 3D point localization/tracking with PSF-engineered event cameras and uses implicit neural representations (SIREN-style MLPs) to design optimal phase and amplitude masks, validated in simulation and a small prototype.
Problem
Point-spread-function engineering has been used for CMOS cameras but not for event cameras; existing PSF designs are not known to be optimal for 3D localization and tracking of moving point sources with event sensors.
Approach
Derives the Fisher information and CRB for event measurements (log-intensity differences of Poisson PSF intensities), then optimizes phase and binary amplitude masks via neural implicit representations (MLPs) to minimize the CRB summed over depths and motion directions.
Key results
- Neural phase mask (NPM) achieves the lowest average CRB (33.1 nm) vs. Fisher phase mask (36.3 nm), open aperture (80.8 nm), and Levin amplitude mask (263.3 nm).
- In 3D tracking, NPM gives 51.2 nm 3D RMSE and 39.2 nm z L1 error, vs. 52.6 nm and 44.2 nm for the Fisher phase mask and 617 nm for open aperture.
- Neural amplitude mask (NAM) reduces CRB to 50.5 nm vs. 263.3 nm for Levin et al. amplitude mask and 65.5 nm for pixel-wise amplitude mask.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NPM instead of the Fisher phase mask | Event-camera 3D tracking · 3µm depth range, all 6 position parameters | CRB (nm): 36.3 -> 33.1 | Helped |
| Use NAM instead of Levin et al.'s amplitude mask | Event-camera 3D tracking · 3µm depth range, all 6 position parameters | CRB (nm): 263.3 -> 50.5 | Helped |
| Use NPM instead of the Fisher phase mask | Event-camera 3D tracking · 3D trajectory recovery | RMSE (nm), 3D: 52.6 -> 51.2 | Helped |
| Use NPM instead of the Fisher phase mask | Event-camera 3D tracking · 3D trajectory recovery | L1 (nm), z: 44.2 -> 39.2 | Helped |
| Use NAM instead of Levin et al.'s amplitude mask | Event-camera 3D tracking · 3D trajectory recovery | RMSE (nm), 3D: 764 -> 66.0 | Helped |
| Use NAM instead of Levin et al.'s amplitude mask | Event-camera 3D tracking · 3D trajectory recovery | L1 (nm), z: 1036 -> 49.2 | Helped |
| Add neural amplitude representation instead of pixel-wise amplitude representation (ablation) | Amplitude-mask optimization · 3µm depth range | CRB (nm): 65.5 -> 50.5 | Helped |
| Add neural phase representation instead of pixel-wise phase representation (ablation) | Phase-mask optimization · 3µm depth range | CRB (nm): 34.2 -> 33.1 | Helped |
| Use neural phase representation instead of Zernike basis representation (ablation) | Phase-mask optimization · 3µm depth range | CRB (nm): 34.8 -> 33.1 | Helped |
| Add neural amplitude representation instead of pixel-wise amplitude representation (ablation) | Amplitude-mask optimization · 3D trajectory recovery | RMSE (nm), 3D: 120 -> 66.0 | Helped |
| Add neural phase representation instead of pixel-wise phase representation (ablation) | Phase-mask optimization · 3D trajectory recovery | RMSE (nm), 3D: 56.5 -> 51.2 | Helped |
| Use neural phase representation instead of Zernike basis representation (ablation) | Phase-mask optimization · 3D trajectory recovery | L1 (nm), z: 50.2 -> 39.2 | Helped |
Surprising findings
- Existing Fisher phase masks designed for CMOS are already near-optimal for localizing static flashing point sources with event cameras.
Limitations
- Binned event frames do not exploit the asynchronous nature of event cameras.
- Bounds assume an idealized event camera with no read noise.
- Only single-emitter images are considered.
Experimental setup
Datasets: simulated Brownian motion trajectories simulated coded event frames
Benchmarks: simulated 3D Brownian motion tracking (5 sequences of 1000 frames)
Metrics: Cramér-Rao bound (nm) 3D RMSE (nm) z L1 error (nm)
Baselines: Open aperture Fisher phase mask Levin et al. amplitude mask Pixel-wise mask Zernike phase mask
Builds on: Fisher PSF (Shechtman et al.) SIREN Levin et al. amplitude mask Zernike phase mask parameterization
Closest papers by meaning
- Event Fields: Capturing Light Fields at High Speed, Resolution, and Dynamic Range (CVPR 2025 · similarity 0.82)
- A Simple and Effective Point-based Network for Event Camera 6-DOFs Pose Relocalization (CVPR 2024 · similarity 0.82)
- Full-DoF Egomotion Estimation for Event Cameras Using Geometric Solvers (CVPR 2025 · similarity 0.81)
- EventSplat: 3D Gaussian Splatting from Moving Event Cameras for Real-time Rendering (CVPR 2025 · similarity 0.81)
- Event-3DGS: Event-based 3D Reconstruction Using 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.81)
- Time-Efficient Light-Field Acquisition Using Coded Aperture and Events (CVPR 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.