Research map / 3D vision / Human pose estimation
EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces EventEgo3D, a lightweight network that regresses real-time 3D human poses from a head-mounted fisheye event camera using LNES inputs and a residual event propagation module, with synthetic and real datasets.
Problem
Monocular egocentric 3D human motion capture with RGB cameras fails under low light and fast motion; the paper targets 3D human pose estimation from an egocentric fisheye event camera on a head-mounted device.
Approach
A U-Net heatmap estimator plus heatmap-to-3D lifting operating on LNES event frames, augmented with a Residual Event Propagation Module that uses a human segmentation mask and confidence map to weight the previous frame's events, emphasizing human-triggered events over background noise.
Key results
- EE3D achieves 107.30 mm MPJPE and 79.66 mm PA-MPJPE on EE3D-R, outperforming Xu et al., Rudnev et al. and Tome et al. (avg MPJPE 133.53, 114.52, 173.01 mm).
- EE3D runs at about 140 Hz pose update rate with 1.25M parameters and 416.84M FLOPs.
- Ablation shows the segmentation decoder and confidence-weighted event propagation reduce MPJPE from 111.01 mm (baseline) to 107.30 mm.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EventEgo3D instead of Rudnev et al. | Egocentric event-based 3D human pose estimation · EE3D-R test (Avg.) | MPJPE: 114.52 -> 107.30 | Helped |
| Use EventEgo3D instead of Rudnev et al. | Egocentric event-based 3D human pose estimation · EE3D-R test (Avg.) | PA-MPJPE: 84.87 -> 79.66 | Helped |
| Add a segmentation decoder (ablation) | EPM-only baseline · EE3D-R | MPJPE: 111.01 -> 108.85 | Helped |
| Add a segmentation decoder (ablation) | EPM-only baseline · EE3D-R | PA-MPJPE: 85.58 -> 84.98 | Helped |
| Add residual event propagation weighted by the segmentation mask, without the confidence decoder (ablation) | Baseline with segmentation decoder · EE3D-R | MPJPE: 108.85 -> 107.58 | Helped |
| Add residual event propagation weighted by the segmentation mask, without the confidence decoder (ablation) | Baseline with segmentation decoder · EE3D-R | PA-MPJPE: 84.98 -> 83.95 | Helped |
| Add the confidence decoder to weight propagated events (ablation) | REPM without confidence decoder · EE3D-R | MPJPE: 107.58 -> 107.30 | Helped |
| Add the confidence decoder to weight propagated events (ablation) | REPM without confidence decoder · EE3D-R | PA-MPJPE: 83.95 -> 79.66 | Helped |
Surprising findings
- Rudnev et al. (EventHands) achieves a higher pose update rate (489.56 Hz) than EE3D, though with lower 3D accuracy.
Limitations
- Real dataset covers only twelve subjects and studio motions; generalisation to in-the-wild scenes is only shown qualitatively.
- Dependence on synthetic pre-training followed by fine-tuning on the real dataset.
- Large errors remain on some motions such as crouching and crawling (MPJPE above 160 mm).
Experimental setup
Datasets: EE3D-S EE3D-R synthetic egocentric renderings from Xu et al.
Benchmarks: EE3D-R
Metrics: MPJPE PA-MPJPE pose update rate FLOPs parameters
Baselines: Xu et al. (egocentric RGB pose) Rudnev et al. (EventHands) Tome et al. (LiftFuse/egocentric RGB pose)
Builds on: LNES (EventHands) Blaze blocks VID2E E2VID
Closest papers by meaning
- 3D Human Pose Perception from Egocentric Stereo Videos (CVPR 2024 · similarity 0.85)
- EF-3DGS: Event-Aided Free-Trajectory 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.85)
- HOT3D: Hand and Object Tracking in 3D from Egocentric Multi-View Videos (CVPR 2025 · similarity 0.85)
- Egocentric Whole-Body Motion Capture with FisheyeViT and Diffusion-Based Motion Refinement (CVPR 2024 · similarity 0.84)
- EAG3R: Event-Augmented 3D Geometry Estimation for Dynamic and Extreme-Lighting Scenes (NeurIPS 2025 · similarity 0.84)
- I'M HOI: Inertia-aware Monocular Capture of 3D Human-Object Interactions (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.