Research map / 3D vision / Human pose estimation
AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AiOS, a one-stage DETR-style transformer that jointly detects multiple humans and regresses SMPL-X whole-body meshes (body, hands, face) from the full image without a separate detector or cropping.
Problem
Expressive human pose and shape estimation (3D whole-body mesh recovery) is usually done in multi-stage top-down pipelines that rely on cropping with detected or ground-truth boxes, which loses context, introduces distractions, and degrades in crowded scenes.
Approach
Builds on DETR with progressive decoders using "Human-as-Tokens": human location tokens give coarse per-person global features, then joint-related tokens add fine-grained local features, with attention masks linking hands/face to their own body, enabling end-to-end multi-person whole-body regression on the full frame.
Key results
- On AGORA test set, AiOS reaches NMVE 97.8 mm vs. 107.2 mm for SMPLer-X (fine-tuned), with better NMJE and detection F1 0.94.
- On AGORA SMPL test set, AiOS0.5 reduces NMVE from 108.3 mm (BEV) to 61.2 mm, a 43% reduction.
- On EHF, AiOS achieves PVE 45.4 mm vs. 61.5 mm for SMPLer-X without ground-truth boxes, a reduction of about 26%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add joint-guided local feature queries to the naive AiOS pipeline (ablation) | AiOS · AGORA validation | PVE All (mm) | Helped |
| Allow tokens to attend to any other tokens (full attention) (ablation) | AiOS · AGORA validation | PVE All (mm): 50.5 -> 54.8 | Hurt |
| Limit hand and face location tokens to attend only to their subject (ablation) | AiOS · AGORA validation | PVE All (mm): 50.5 -> 52.8 | Hurt |
| Apply SMPL-X supervision at every stage (ablation) | AiOS · AGORA validation | PVE All (mm): 50.5 -> 55.7 | Hurt |
| Apply SMPL-X supervision only at the third stage (ablation) | AiOS · AGORA validation | PVE All (mm): 50.5 -> 51.8 | Hurt |
| Use AiOS instead of SMPLer-X on AGORA SMPL-X test | AGORA SMPL-X test · AGORA SMPL-X test | NMVE (mm): 107.2 -> 97.8 | Helped |
| Use AiOS instead of BEV on AGORA SMPL test | AGORA SMPL test · AGORA SMPL test | NMVE (mm): 108.3 -> 61.2 | Helped |
| Use AiOS instead of SMPLer-X on UBody | UBody · UBody | PVE All (mm): 61.5 -> 58.6 | Helped |
| Use AiOS instead of SMPLer-X on EHF | EHF · EHF | PVE All (mm): 65.4 -> 45.4 | Helped |
Surprising findings
- Replacing ground-truth boxes with AiOS's own estimated boxes improved OSX and SMPLer-X whole-body MVE on AGORA, suggesting cropped two-stage methods are sensitive to box quality.
Limitations
- Performance gains are limited by the available multi-person real data for training.
- Hand estimation under limited resolution remains challenging.
- Does not yet extend to tracking or 3D localization.
Experimental setup
Datasets: AGORA BEDLAM COCO UBody ARCTIC EgoBody
Benchmarks: AGORA UBody EHF ARCTIC EgoBody BEDLAM
Metrics: PA-MPVPE MPVPE MVE MPJPE NMVE NMJE F1-score precision recall PA-PVE
Baselines: OSX SMPLer-X Hand4Whole PyMAF-X HybrIK-X BEDLAM ROMP BEV PIXIE
Builds on: DETR DAB-DETR ED-Pose ROMP SMPL-X
Closest papers by meaning
- End-to-End Recovery of Human Shape and Pose (CVPR 2018 · similarity 0.90)
- FactorizedHMR: A Hybrid Framework for Video Human Mesh Recovery (NeurIPS 2026 · similarity 0.87)
- BLADE: Single-view Body Mesh Estimation through Accurate Depth Estimation (CVPR 2025 · similarity 0.87)
- TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation (CVPR 2024 · similarity 0.86)
- HiPoser: 3D Human Pose Estimation with Hierarchical Shared Learning at Parts-Level Using Inertial Measurement Units (AAAI 2025 · similarity 0.86)
- SAT-HMR: Real-Time Multi-Person 3D Mesh Estimation via Scale-Adaptive Tokens (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.