Research map / 3D vision / Human pose estimation
A Benchmark Dataset for Event-Guided Human Pose Estimation and Tracking in Extreme Conditions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces EHPT-XC, a real-captured RGB-event benchmark with 2D pose and track annotations for multi-person pose estimation and tracking under low-light and motion blur, using a triplet camera rig and a baseline RGB+event fusion module.
Problem
Multi-person pose estimation and tracking degrade severely under motion blur and low-light conditions, and no real-captured multi-human pose/tracking dataset with paired event data exists for these extreme conditions.
Approach
A triplet camera system (two RGB cameras plus one event camera aligned by beam splitters and hardware-synchronized) captures degraded RGB and event data alongside a sharp, well-lit RGB reference used only for annotation, enabling accurate pose labels on real blurred and low-light frames.
Key results
- On EHPT-XC, DEKR with RGB+event fusion reaches 37.3 mAP@0.5:0.95 versus 25.1 for RGB-only DEKR.
- Event-only HigherHRNet achieves 32.1 mAP and 83.8 mAR, far above RGB-only HigherHRNet (22.7 mAP, 67.0 mAR).
- For tracking, DEKR+ByteTrack improves MOTA from 33.19 (RGB) to 47.37 (RGB+event).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add event data and use the proposed fusion | HigherHRNet · EHPT-XC test | mAP@0.5:0.95: 22.7 -> 34.3 | Helped |
| Add event data and use the proposed fusion | DEKR · EHPT-XC test | mAP@0.5:0.95: 25.1 -> 37.3 | Helped |
| Add event data and use the proposed fusion | CID · EHPT-XC test | mAP@0.5:0.95: 24.0 -> 34.5 | Helped |
| Add event data using concatenation | DEKR + ByteTrack · EHPT-XC test | MOTA: 33.19 -> 47.37 (+14.18) | Helped |
| Add event data using concatenation | DEKR + UniTrack · EHPT-XC test | MOTA: 25.60 -> 46.82 (+21.22) | Helped |
| Add event data using concatenation | DEKR + OC-SORT · EHPT-XC test | MOTA: 23.25 -> 42.34 (+19.09) | Helped |
| Use the proposed fusion instead of prior fusion [29] | HigherHRNet with RGB + Event · EHPT-XC test | mAP@0.5:0.95: 33.8 -> 34.3 | Helped |
| Use the proposed fusion instead of prior fusion [29] | DEKR with RGB + Event · EHPT-XC test | mAP@0.5:0.95: 36.9 -> 37.3 | Helped |
| Use the proposed fusion instead of prior fusion [29] | CID with RGB + Event · EHPT-XC test | mAP@0.5:0.95: 33.7 -> 34.5 | Helped |
Surprising findings
- Event-only pose estimation outperforms RGB-only pose estimation under the extreme conditions studied.
- OC-SORT performs worst among trackers, attributed to large inter-frame displacement that makes long-range association unhelpful.
Limitations
- The proposed multi-modal fusion baseline is simple; more sophisticated fusion may be needed.
- Multi-person tracking evaluation uses only detections from DEKR; event modality increases false positives.
- Dataset scale (158 sequences, 16K images) is modest relative to standard pose benchmarks.
Experimental setup
Datasets: EHPT-XC
Benchmarks: EHPT-XC
Metrics: mAP mAR MOTA IDF1 FP IDSW FN
Baselines: HigherHRNet DEKR CID ByteTrack UniTrack OC-SORT
Builds on: DEKR CID HigherHRNet ByteTrack voxel grid
Closest papers by meaning
- Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baseline (CVPR 2024 · similarity 0.84)
- ETAP: Event-based Tracking of Any Point (CVPR 2025 · similarity 0.83)
- Towards Robust Event-guided Low-Light Image Enhancement: A Large-Scale Real-World Event-Image Dataset and Novel Approach (CVPR 2024 · similarity 0.83)
- HumanMM: Global Human Motion Recovery from Multi-shot Videos (CVPR 2025 · similarity 0.83)
- AE-NeRF: Augmenting Event-Based Neural Radiance Fields for Non-ideal Conditions and Larger Scenes (AAAI 2025 · similarity 0.83)
- LED: A Large-scale Real-world Paired Dataset for Event Camera Denoising (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.