Research map / Detection and segmentation / Tracking
Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baseline
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HDETrack, a hierarchical knowledge distillation framework that transfers RGB-event/multi-view knowledge from a Transformer teacher to an event-only student tracker, and releases EventVOT, a 1280×720 event tracking benchmark.
Problem
Event-based visual object tracking suffers from sparse, noisy, low-resolution event data, while RGB-event fusion trackers incur higher inference cost; there is also no high-resolution event tracking benchmark.
Approach
Train a multi-modal (RGB+event) or multi-view (event image+voxel) Transformer teacher, then distill its similarity matrices, token features, and response maps into an event-only student Transformer so that only event data is needed at test time.
Key results
- On EventVOT, the distilled event-only tracker reaches SR 57.8, PR 62.2, NPR 73.5 versus 55.4/60.4/71.1 for its OSTrack baseline, running at 105 FPS.
- On FE240hz, the method achieves 59.8 SR / 92.2 PR, above OSTrack (57.1/89.3) and STNet (58.5/89.6).
- On COESOT, the method reaches SR 53.1 and PR 64.1, versus 50.9/61.8 for the base tracker; component ablation shows all three distillation losses help.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HDETrack instead of OSTrack | event-based tracking · EventVOT | SR: 55.4 -> 57.8 | Helped |
| use HDETrack instead of OSTrack | event-based tracking · FE240hz | SR: 57.1 -> 59.8 | Helped |
| use event-only HDETrack instead of CEUTrack with RGB and event input | visual tracking · VisEvent | SR: 64.89 -> 37.3 | Hurt |
| use HDETrack instead of DiMP50 | event-based tracking · COESOT | SR: 53.8 -> 53.1 | Hurt |
| add similarity-matrix distillation (SKD) (ablation) | event tracker · COESOT | PR/SR: 61.8/50.9 -> 62.5/51.6 | Helped |
| add similarity-matrix distillation (SKD) (ablation) | event tracker · EventVOT | PR/SR: 60.4/55.4 -> 60.8/56.5 | Helped |
| add feature-based distillation (FKD) (ablation) | event tracker · COESOT | PR/SR: 61.8/50.9 -> 63.0/52.1 | Helped |
| add feature-based distillation (FKD) (ablation) | event tracker · EventVOT | PR/SR: 60.4/55.4 -> 60.4/56.4 | Mixed |
| add response-map distillation (RKD) (ablation) | event tracker · COESOT | PR/SR: 61.8/50.9 -> 62.3/51.5 | Helped |
| add response-map distillation (RKD) (ablation) | event tracker · EventVOT | PR/SR: 60.4/55.4 -> 60.6/56.2 | Helped |
| add SKD, FKD, and RKD together (ablation) | event tracker · COESOT | PR/SR: 61.8/50.9 -> 64.1/53.1 | Helped |
| add SKD, FKD, and RKD together (ablation) | event tracker · EventVOT | PR/SR: 60.4/55.4 -> 62.2/57.8 | Helped |
Surprising findings
- Event voxel input performs very poorly on the high-resolution EventVOT data (SR 8.6), far below event frames (57.8).
- On VisEvent, the event-only method is compared against RGB+event fusion trackers, which remain stronger (CEUTrack 64.89 SR).
Limitations
- Event voxel representation performs poorly on high-resolution event streams, indicating representation design is unresolved.
- Benchmarked trackers were retrained on EventVOT rather than tested with original weights, so comparisons depend on the authors' training setup.
- Only 18 validation videos and small per-category sample counts in EventVOT.
Experimental setup
Datasets: EventVOT FE240hz VisEvent COESOT
Benchmarks: EventVOT FE240hz VisEvent COESOT
Metrics: Success Rate (SR) Precision Rate (PR) Normalized Precision Rate (NPR) FPS
Baselines: OSTrack TrDiMP ToMP50 AiATrack STARK TransT DiMP50 PrDiMP KYS MixFormer ATOM SimTrack
Builds on: OSTrack CEUTrack GWF focal loss (CenterNet-style)
Closest papers by meaning
- PASS: Path-selective State Space Model for Event-based Recognition (NeurIPS 2025 · similarity 0.87)
- Single-Model and Any-Modality for Video Object Tracking (CVPR 2024 · similarity 0.87)
- EventZoom: A Progressive Approach to Event-Based Data Augmentation for Enhanced Neuromorphic Vision (AAAI 2025 · similarity 0.86)
- E2HQV: High-Quality Video Generation from Event Camera via Theory-Inspired Model-Aided Deep Learning (AAAI 2024 · similarity 0.86)
- Fully Spiking Neural Networks for Unified Frame-Event Object Tracking (NeurIPS 2025 · similarity 0.86)
- Revisiting motion information for RGB-Event tracking with MOT philosophy (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.