Research map / Detection and segmentation / CNN features
SD2Event:Self-supervised Learning of Dynamic Detectors and Contextual Descriptors for Event Cameras
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SD2Event, a self-supervised event-camera keypoint detector that learns contextual descriptors via agent-based attention and dynamic detectors, trained from scratch with camera-pose-warped event frame pairs.
Problem
Event-based keypoint detection lacks ground-truth keypoints, CNN descriptors have limited receptive fields and are sensitive to noise, and fixed detectors cannot adapt to diverse keypoint patterns.
Approach
Uses agent-based attention (learnable descriptor agents that aggregate long-range context and select the most similar agent per feature) to build noise-robust contextual descriptors, and generates input-dependent dynamic detectors from detector prototypes refined by descriptor agents, supervised by a self-supervised cosine similarity loss on pose-warped event frame pairs.
Key results
- On the Event-Camera dataset, SD2Event reaches 1.64 px reprojection error vs. 2.16 for SILC (best baseline in the table).
- On N-Caltech101, feature matching IoU is 0.89 vs. 0.83 for EventPoint.
- On HVGA ATIS Corner at 25/50/100 ms, SD2Event gets 0.67/0.76/0.93 px vs. 1.27/1.41/1.72 px for EventPoint.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SD2Event instead of SILC | event keypoint detection · Event-Camera | reprojection error (pixels): 2.16 -> 1.64 | Helped |
| Use SD2Event instead of EventPoint | event keypoint detection · N-Caltech101 | feature matching IoU: 0.83 -> 0.89 | Helped |
| Use SD2Event instead of EventPoint | event keypoint detection · HVGA ATIS Corner | reprojection error (pixels): 1.27 -> 0.67; 1.41 -> 0.76; 1.72 -> 0.93 | Helped |
| Add CFDL while retaining a fixed 1×1 convolutional detector (ablation) | SD2Event · HVGA ATIS Corner (25ms) | reprojection error (pixels): 5.45 -> 1.13 | Helped |
| Add CFDL while retaining a fixed 1×1 convolutional detector (ablation) | SD2Event · HVGA ATIS Corner (50ms) | reprojection error (pixels): 10.34 -> 1.27 | Helped |
| Add CFDL while retaining a fixed 1×1 convolutional detector (ablation) | SD2Event · HVGA ATIS Corner (100ms) | reprojection error (pixels): 16.99 -> 1.54 | Helped |
| Add DKDL to original features (ablation) | SD2Event · HVGA ATIS Corner (25ms) | reprojection error (pixels): 5.45 -> 0.89 | Helped |
| Add DKDL to original features (ablation) | SD2Event · HVGA ATIS Corner (50ms) | reprojection error (pixels): 10.34 -> 1.01 | Helped |
| Add DKDL to original features (ablation) | SD2Event · HVGA ATIS Corner (100ms) | reprojection error (pixels): 16.99 -> 1.25 | Helped |
| Increase descriptor agents from 2 to 32 (ablation) | SD2Event CFDL · HVGA ATIS Corner (25ms) | reprojection error (pixels): 0.85 -> 0.67 | Helped |
| Increase descriptor agents from 32 to 64 (ablation) | SD2Event CFDL · HVGA ATIS Corner (25ms) | reprojection error (pixels): 0.67 -> 0.68 | Hurt |
| Increase detector prototypes from 2 to 16 (ablation) | SD2Event DKDL · HVGA ATIS Corner (25ms) | reprojection error (pixels): 0.96 -> 0.67 | Helped |
Surprising findings
- Full attention aggregates noise from event streams, which agent-based attention avoids according to the authors' qualitative analysis.
Limitations
- Ablations and tuning are performed only on the HVGA ATIS Corner dataset.
- Full attention is cited as costly, but no detailed efficiency comparison with other baselines is reported in the main text.
Experimental setup
Datasets: Event-Camera dataset (training: outdoor dataset [12], indoor dataset [21]) N-Caltech101 HVGA ATIS Corner
Benchmarks: Event-Camera dataset N-Caltech101 HVGA ATIS Corner
Metrics: reprojection error (pixels) feature matching IoU reprojection error across 25ms/50ms/100ms intervals
Baselines: EventPoint SILC evHarris evFast Arc luvHarris DART Chiberre et al.
Builds on: SuperPoint EventPoint Tencode Time Surface Transformer attention
Closest papers by meaning
- LEOD: Label-Efficient Object Detection for Event Cameras (CVPR 2024 · similarity 0.86)
- PASS: Path-selective State Space Model for Event-based Recognition (NeurIPS 2025 · similarity 0.84)
- FlexEvent: Towards Flexible Event-Frame Object Detection at Varying Operational Frequencies (NeurIPS 2025 · similarity 0.84)
- EventZoom: A Progressive Approach to Event-Based Data Augmentation for Enhanced Neuromorphic Vision (AAAI 2025 · similarity 0.83)
- Flow-Event Autoencoder: Event Stream Object Recognition Dataset Generation with Arbitrary High Temporal Resolution (AAAI 2024 · similarity 0.83)
- E2HQV: High-Quality Video Generation from Event Camera via Theory-Inspired Model-Aided Deep Learning (AAAI 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.