Research map / Video / Video segmentation and tracking
Low-power Continuous Remote Behavioral Localization with Event Cameras
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces event-camera-based temporal action detection of penguin "ecstatic display" behavior, with a robust event-rate proposal generator (reTAG) and a ResNet18-based augmented temporal segment classifier (ATSN), plus a new Antarctic event dataset.
Problem
Automatically quantifying animal behavior (ecstatic display in Chinstrap penguins) at remote wild locations under power, storage, and variable lighting constraints, formulated as temporal action detection from event camera data.
Approach
Uses the event rate as an actioness score to generate temporal proposals (robust-normalized, multi-threshold watershed plus merging, extending TAG), then classifies augmented proposals with a sparsely sampled, event-histogram/time-map based ResNet18 segment network.
Key results
- Full pipeline (time-map ATSN) reaches mAP of 58% averaged over tIoU 0.1-0.7, outperforming event-adapted ActionFormer (55%) and bottom-up CNN (45%).
- Undisturbed daytime mAP rises to 63%, night mAP is 62%, while snow/precipitation drops to 37%.
- reTAG proposals achieve AR of 0.66 with top-50 proposals versus 0.49 for event TAG; DAVIS346 uses 0.83W while recording vs 2.44W for a frame camera.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use robust event-rate normalization in reTAG instead of event TAG | Temporal proposal generation · Penguin ED test set, Top 50 proposals | average recall: 0.49 -> 0.66 | Helped |
| Use the full time-map proposal-and-classification pipeline instead of ActionFormer | Penguin ED detection · Penguin ED test set | mAP (average): 0.55 -> 0.58 | Helped |
| Use time maps instead of histograms | ATSN · Penguin ED test set | mAP (average): 0.56 -> 0.58 | Helped |
| Use event-based detection in night conditions rather than undisturbed daylight | Ours + time-map · Penguin ED test set | mAP (average): 0.63 -> 0.62 | No clear effect |
| Detect in precipitation rather than undisturbed daylight | Ours + time-map · Penguin ED test set | mAP (average): 0.63 -> 0.37 | Hurt |
| Add proposal augmentation (ablation) | ATSN · Penguin ED test set | mAP (average): 0.17 -> 0.58 | Helped |
| Use 1 central timestamp sample instead of 7 (ablation) | ATSN · Penguin ED test set | mAP (average): 0.58 -> 0.40 | Hurt |
| Use 3 central timestamp samples instead of 7 (ablation) | ATSN · Penguin ED test set | mAP (average): 0.58 -> 0.50 | Hurt |
| Use 5 central timestamp samples instead of 7 (ablation) | ATSN · Penguin ED test set | mAP (average): 0.58 -> 0.55 | Hurt |
| Use 9 central timestamp samples instead of 7 (ablation) | ATSN · Penguin ED test set | mAP (average): 0.58 -> 0.56 | Hurt |
| Reduce augmentation width to 20% (ablation) | Ours + histogram · Penguin ED test set | mAP (average): 0.58 -> 0.52 | Hurt |
| Increase augmentation width to 50% (ablation) | Ours + histogram · Penguin ED test set | mAP (average): 0.58 -> 0.54 | Hurt |
Surprising findings
- Grayscale frames are unusable at night, while event-based detection performance at night is comparable to daytime.
Limitations
- Cannot record during heavy weather; strong wind causes vibrations and noisy events.
- Assumes high event rate correlates with the behavior of interest, limiting generalization.
- Requires user-supplied regions of interest (bounding boxes) per nest.
Experimental setup
Datasets: Chinstrap penguin event dataset (DAVIS346, Antarctica) ImageNet
Benchmarks: Chinstrap penguin ecstatic display event dataset
Metrics: mean average precision (mAP) at tIoU 0.1/0.3/0.5/0.7 average recall (AR)
Baselines: ActionFormer (event-adapted) Bottom-up CNN (w/o and w/ morphological filter) Sliding Window Watershed event TAG
Builds on: TAG (temporal actioness grouping) Temporal Segment Network ActionFormer
Closest papers by meaning
- MammAlps: A Multi-view Video Behavior Monitoring Dataset of Wild Mammals in the Swiss Alps (CVPR 2025 · similarity 0.85)
- The PanAf-FGBG Dataset: Understanding the Impact of Backgrounds in Wildlife Behaviour Recognition (CVPR 2025 · similarity 0.85)
- BuckTales: A multi-UAV dataset for multi-object tracking and re-identification of wild antelopes (NeurIPS 2024 · similarity 0.82)
- MOVEBENCH: A Benchmark for Global-Scale Wildlife Movement Forecasting (NeurIPS 2026 · similarity 0.80)
- ETAP: Event-based Tracking of Any Point (CVPR 2025 · similarity 0.80)
- TimeTracker: Event-based Continuous Point Tracking for Video Frame Interpolation with Non-linear Motion (CVPR 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.