Research map / 3D vision / Human pose estimation
MVDoppler-Pose: Multi-Modal Multi-View mmWave Sensing for Long-Distance Self-Occluded Human Walking Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MVDoppler-Pose, a multi-modal (range-time and Doppler-time) multi-view mmWave radar model with cross-modal and cross-view sparse-attention fusion and a motion-aware loss, for occlusion- and distance-resilient 3D walking pose estimation, plus an expanded MVDoppler-Pose dataset.
Problem
3D human walking pose estimation suffers from self-occlusion and distance-dependent degradation for cameras, while mmWave radar has directional sensitivity (poor lateral resolution and Doppler quality depending on motion direction) that limits existing mmWave pose methods to controlled movements.
Approach
Fuse positional (range-time) and motional (Doppler-time) mmWave representations within each sensor via cross-modal fusion, then fuse across two orthogonal radar views via cross-view fusion, using sparse cross-attention inside MobileViT blocks, trained with a loss that adds a Pearson-correlation-based motion-distance term on limb joints.
Key results
- Multi-view mmWave-multi model achieves 58.24 to 63.27 mm MPJPE across distance levels, versus 88.32 to 137.16 mm for RGB baselines.
- Single-view mmWave fusion model reaches 96.59 mm MPJPE at <8.5m, while vision baselines degrade with distance, and beyond 8.5m the single-view mmWave model outperforms vision.
- Cross-domain loss improves the multi-modal model from 67.53 to 60.96 mm MPJPE and PCK from 89.37 to 93.24%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use multi-view positional and motional inputs instead of single-view inputs | MVDoppler-Pose · Self-occluded walking, across the three distance bands | MPJPE: 108.09 -> 70.10; 102.95 -> 67.22; 105.93 -> 64.83 | Helped |
| combine positional and motional modalities | MVDoppler-Pose (Single-View) · Self-occluded walking, across the three distance bands | MPJPE: 108.09 -> 96.59; 102.95 -> 89.35; 105.93 -> 88.32 | Helped |
| use multi-view positional and motional inputs instead of single-view inputs | MVDoppler-Pose · Self-occluded walking, across the three distance bands | MPJPE: 96.59 -> 63.27; 89.35 -> 59.11; 88.32 -> 58.24 | Helped |
| add motional information to positional input | MVDoppler-Pose (Multi-View) · Self-occluded walking, across the three distance bands | MPJPE: 70.10 -> 63.27; 67.22 -> 59.11; 64.83 -> 58.24 | Helped |
| use multi-layered fusion instead of single-layer fusion (ablation) | MVDoppler-Pose · MVDoppler-Pose multi-view fusion | MPJPE: 65.09 -> 60.96 | Helped |
| use multi-layered fusion instead of single-layer fusion (ablation) | MVDoppler-Pose · MVDoppler-Pose multi-view fusion | PCK: 90.65 -> 93.24 | Helped |
| add sparse attention (ablation) | MVDoppler-Pose · MVDoppler-Pose multi-view fusion | MPJPE: 62.77 -> 60.96 | Helped |
| add sparse attention (ablation) | MVDoppler-Pose · MVDoppler-Pose multi-view fusion | PCK: 91.17 -> 93.24 | Helped |
| add sparse attention (ablation) | MVDoppler-Pose · MVDoppler-Pose multi-view fusion | ρ: 0.52 -> 0.53 | Helped |
| replace positional loss with cross-domain loss (ablation) | MVDoppler-Pose (mmW-Multi) · MVDoppler-Pose ablation evaluation | MPJPE: 67.53 -> 60.96 | Helped |
| replace positional loss with cross-domain loss (ablation) | MVDoppler-Pose (mmW-Multi) · MVDoppler-Pose ablation evaluation | PCK: 89.37 -> 93.24 | Helped |
| replace positional loss with cross-domain loss (ablation) | MVDoppler-Pose (mmW-Multi) · MVDoppler-Pose ablation evaluation | ρ: 0.45 -> 0.53 | Helped |
Surprising findings
- Single-view mmWave models are notably weaker when subjects walk tangentially to the radar (around ±90°), while the multi-view setup removes this degradation.
Limitations
- Evaluated only with a single subject and minimal clutter scenarios.
- Limited range of walking activities.
- Data collected only in the MVDoppler dataset setting; no in-the-wild validation.
Experimental setup
Datasets: MVDoppler-Pose
Benchmarks: MVDoppler-Pose
Metrics: MPJPE PCK@150mm temporal correlation (Pearson)
Baselines: Sun et al. (HRNet-style 3D pose from RGB) Wei et al. MHFormer Zhao et al. frequency-based 3D pose Wang et al. mmWave position model Zhou et al. Doppler-based pose Tang et al. position-motion pose Wang et al. fusion Prakash et al. fusion Li et al. fusion
Builds on: MobileViT MVDoppler sparse attention MHFormer
Closest papers by meaning
- Doppler Prompting for Stable mmWave-based Human Pose Estimation (ICML 2026 · similarity 0.89)
- MITO: A Millimeter-Wave Dataset and Simulator for Non-Line-of-Sight Perception (NeurIPS 2026 · similarity 0.83)
- RAPTR: Radar-based 3D Pose Estimation using Transformer (NeurIPS 2025 · similarity 0.82)
- HGSFusion: Radar-Camera Fusion with Hybrid Generation and Synchronization for 3D Object Detection (AAAI 2025 · similarity 0.82)
- mmWalk: Towards Multi-modal Multi-view Walking Assistance (NeurIPS 2025 · similarity 0.80)
- UMotion: Uncertainty-driven Human Motion Estimation from Inertial and Ultra-wideband Units (CVPR 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.