Research map / 3D vision / Human pose estimation
Sound-based Multi-Person 3D Pose Estimation
Yusuke Oumi, Yuto Shibata, Go Irie, Akisato Kimura, Yoshimitsu Aoki, Mariko Isogawa
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Acoustic Multi-scale Encoder and Temporal Pose Decoder (full SoundMHPE) versus adapted baselines | SoundMHPE · AMP dataset (cross-group, 3-fold) | MPJPE, PA-MPJPE, PCK | Improved |
| add Temporal Pose Decoder (one query per predicted frame per person) (ablation) | SoundMHPE · AMP dataset | MPJPE, PA-MPJPE, PCK | Improved |
| add Acoustic Multi-scale Encoder (multi-window STFT) instead of a single-window spectrogram (ablation) | SoundMHPE · AMP dataset | MPJPE, PA-MPJPE, PCK | Improved |
| replace standard self-attention with Temporal and Frequency Self-Attention in the encoder (ablation) | SoundMHPE encoder · AMP dataset | MPJPE, PA-MPJPE, PCK | Improved |
| replace standard self-attention with Motion and Interaction Self-Attention in the decoder (ablation) | SoundMHPE decoder · AMP dataset | MPJPE, PA-MPJPE, PCK | Improved |
| use window sizes (L, 2L, 4L) instead of (L/2, L, 2L), (L/4, L/2, L) or (L, 2L) (ablation) | Acoustic Multi-scale Encoder · AMP dataset | accuracy | Improved |
| use finer temporal windows at the expense of frequency resolution (ablation) | Acoustic Multi-scale Encoder · AMP dataset | accuracy | Hurt |
| apply AME attention and TPD to WiFi CSI features instead of PiW (Single) | Person-in-WiFi 3D · PiW dataset | pose accuracy | Improved |
| extend single-frame PiW to predict a 0.27-s sequence (T=4) without TPD | PiW · PiW dataset | pose accuracy | Hurt |
| move from single-person to three-person scenes | SoundMHPE · AMP dataset (1 vs 3 persons) | pose accuracy | No clear effect |
Closest papers by meaning
- Listening Human Behavior: 3D Human Pose Estimation With Acoustic Signals (CVPR 2023 · similarity 0.91)
- Dual Networks Based 3D Multi-Person Pose Estimation From Monocular Video (TPAMI 2022 · similarity 0.89)
- Unsupervised Cross-Modal Alignment for Multi-Person 3D Pose Estimation (ECCV 2020 · similarity 0.89)
- Monocular 3D Pose and Shape Estimation of Multiple People in Natural Scenes - The Importance of Multiple Scene Constraints (CVPR 2018 · similarity 0.89)
- Shape-Aware Multi-Person Pose Estimation From Multi-View Images (ICCV 2021 · similarity 0.88)
- Motions as Queries: One-Stage Multi-Person Holistic Human Motion Capture (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
