Research map / Video / Video segmentation and tracking
Recognizing Ultra-High-Speed Moving Objects with Bio-Inspired Spike Camera
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes spiking motion blur in spike cameras theoretically and proposes a temporal-spatial context spiking representation (TSC) with multi-span dilated temporal convolution, residual deformable convolution, and cross-attention fusion, plus a new real-captured ultra-high-speed recognition dataset (UHSR).
Problem
Spike cameras' CMOS physical limits cause spiking motion blur for objects moving faster than roughly 500 km/h, degrading recognition of ultra-high-speed moving objects; existing spike datasets are synthetic or not ultra-high-speed.
Approach
Theoretical analysis attributes spiking motion blur to temporal undersampling and spatial misalignment, then a representation learns temporal context via multi-span dilated convolutions with learnable re-weighting and spatial context via cascaded residual deformable convolutions, fused by cross-attention.
Key results
- TSC achieves 73.2% accuracy on U-CIF (10 classes) and 67.5% on U-CAL (101 classes) on UHSR, outperforming DMER (67.8%, 62.4%) and other spike-based methods.
- On SpiReco, TSC reaches 60.8% on S-CIF and 74.6% on S-CAL, versus 57.9% and 71.2% for DMER.
- Ablation shows the ResNet-18 baseline improves from 65.7% to 73.2% on U-CIF and from 59.2% to 67.5% on U-CAL with both modules.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TSC instead of DMER | Spike-based recognition with ResNet-18 · S-CIF | accuracy: 57.9% -> 60.8% | Helped |
| Use TSC instead of DMER | Spike-based recognition with ResNet-18 · S-CAL | accuracy: 71.2% -> 74.6% | Helped |
| Use TSC instead of DMER | Spike-based recognition with ResNet-18 · U-CIF | accuracy: 67.8% -> 73.2% | Helped |
| Use TSC instead of DMER | Spike-based recognition with ResNet-18 · U-CAL | accuracy: 62.4% -> 67.5% | Helped |
| Add MTDC module (ablation) | TSC · S-CIF | accuracy: 55.1% -> 57.6% | Helped |
| Add RDC module (ablation) | TSC · S-CIF | accuracy: 55.1% -> 58.9% | Helped |
| Add both MTDC and RDC modules (ablation) | TSC · S-CIF | accuracy: 55.1% -> 60.8% | Helped |
| Add MTDC module (ablation) | TSC · S-CAL | accuracy: 69.3% -> 71.7% | Helped |
| Add RDC module (ablation) | TSC · S-CAL | accuracy: 69.3% -> 73.6% | Helped |
| Add MTDC module (ablation) | TSC · U-CIF | accuracy: 65.7% -> 68.3% | Helped |
| Add RDC module (ablation) | TSC · U-CIF | accuracy: 65.7% -> 70.5% | Helped |
| Add MTDC module (ablation) | TSC · U-CAL | accuracy: 59.2% -> 63.4% | Helped |
Surprising findings
- Spiking motion blur onset is tied to scene brightness: lower luminance lowers the speed upper bound V*, making blur more prominent.
Limitations
- Spike-based SNN and event-based baselines were trained under the same setup, which may not reflect their best performance; SNNs showed optimization difficulties.
- Dataset speeds are equivalent (500-700 km/h) from a motion platform in a laboratory setting rather than fully natural scenes.
- Ultra-high-speed analysis relies on a single camera-object geometry and a constant eta derived from sensor parameters.
Experimental setup
Datasets: UHSR SpiReco CIFAR-10 Caltech-101
Benchmarks: UHSR (U-CIF, U-CAL) SpiReco (S-CIF, S-CAL)
Metrics: recognition accuracy Two-Dimensional Entropy (TDE) Blind Image Quality Index (BIQI)
Baselines: TDBN SEWR DSR EtoF BEI SBNE TSR TFP ISI DMER
Builds on: ISI spiking representation TSR DMER Deformable convolution
Closest papers by meaning
- Finding Visual Saliency in Continuous Spike Stream (AAAI 2024 · similarity 0.87)
- Optical Flow for Spike Camera with Hierarchical Spatial-Temporal Spike Fusion (AAAI 2024 · similarity 0.86)
- Boosting Spike Camera Image Reconstruction from a Perspective of Dealing with Spike Fluctuations (CVPR 2024 · similarity 0.86)
- SpikeReveal: Unlocking Temporal Sequences from Real Blurry Inputs with Spike Streams (NeurIPS 2024 · similarity 0.86)
- Spatio-Temporal Interactive Learning for Efficient Image Reconstruction of Spiking Cameras (NeurIPS 2024 · similarity 0.86)
- SFOD: Spiking Fusion Object Detector (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.