Research map / 3D vision / Human pose estimation
EgoMo3R: Joint Egocentric Motion and Scene Reconstruction
Chaitanya Patel, Yusuke Kato, Rin Takahira, Kazuki Kozuka, Juan Carlos Niebles, Ehsan Adeli
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EgoMo3R, a method that jointly reconstructs egocentric 3D human motion and the surrounding scene from egocentric video, rather than treating the two tasks separately.
Problem
Egocentric 3D human motion reconstruction usually assumes metric, gravity-aligned camera trajectories from inertial sensors, and scene reconstruction is handled separately even though the two tasks are coupled.
Approach
Treats egocentric motion and scene reconstruction as a single coupled problem solved jointly from video, removing the need for specialized inertial sensors to obtain metric, gravity-aligned camera trajectories.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add point-track loss to flow-only optimization (ablation) | EgoMo3R scene-only · EgoExo4D locomotion | ATE: 0.604 -> 0.361 | Improved |
| add depth-scale regularizer L_depth (ablation) | EgoMo3R scene-only · EgoExo4D locomotion | ATE: 0.361 -> 0.353 | Improved |
| add floor coplanarity loss L_scene (ablation) | EgoMo3R scene-only · EgoExo4D locomotion | ATE: 0.353 -> 0.371 (Acc mean 0.414 -> 0.398) | Mixed |
| add motion prior with direct joint optimization (ablation) | EgoMo3R · EgoExo4D locomotion | MPJPE: MPJPE 0.456 | Hurt |
| use alternating instead of direct optimization of motion prior (ablation) | EgoMo3R · EgoExo4D locomotion | MPJPE: 0.456 -> 0.320 | Improved |
| use alternating instead of direct optimization (ablation) | EgoMo3R · EgoExo4D locomotion | ATE: 0.350 -> 0.318 | Improved |
| replace motion prior with camera-smoothness prior (ablation) | EgoMo3R · EgoExo4D locomotion | ATE: 0.318 -> 0.335 | Hurt |
| replace motion prior with camera-smoothness prior (ablation) | EgoMo3R · EgoExo4D locomotion | RPE Trans: 0.095 -> 0.074 | Improved |
| add a third alternation round (ablation) | EgoMo3R · EgoExo4D locomotion | ATE: 0.318 -> 0.300 | Improved |
| warm-start camera trajectory with flow-only optimization before full objective | EgoMo3R · EgoExo4D | trajectory stability | Improved |
| use EgoMo3R joint optimization instead of CUT3R/ViPE trajectory + EgoAllo | EgoMo3R · EgoExo4D | MPJPE: 0.348 -> 0.271 | Improved |
| use weaker point tracker or depth model (ablation) | EgoMo3R scene-only · EgoExo4D locomotion | ATE | Hurt |
Closest papers by meaning
- EgoPoseFormer v2: Accurate Egocentric Human Motion Estimation for AR/VR (CVPR 2026 · similarity 0.91)
- EgoGen: An Egocentric Synthetic Data Generator (CVPR 2024 · similarity 0.90)
- EgoSim: An Egocentric Multi-view Simulator and Real Dataset for Body-worn Cameras during Motion and Activity (NeurIPS 2024 · similarity 0.90)
- FRAME: Floor-aligned Representation for Avatar Motion from Egocentric Video (CVPR 2025 · similarity 0.90)
- EgoVid-5M: A Large-Scale Video-Action Dataset for Egocentric Videos Generation (NeurIPS 2025 · similarity 0.89)
- 3D Human Pose Perception from Egocentric Stereo Videos (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
