Research map / 3D vision / Human pose estimation
SAT-HMR: Real-Time Multi-Person 3D Mesh Estimation via Scale-Adaptive Tokens
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAT-HMR, a one-stage DETR-style multi-person 3D human mesh estimator that uses scale-adaptive tokens (high resolution for small people, pooled background) to reach real-time speed with near-SOTA accuracy.
Problem
Existing one-stage DETR-style multi-person human mesh methods need high-resolution inputs for accuracy, especially for small-scale individuals, but this makes them computationally expensive and not real-time.
Approach
A patch-level scale map predicted from low-resolution tokens decides which patches are replaced by high-resolution tokens (for small-scale people), which large-scale patches are kept at low resolution, and which background patches are pooled, yielding scale-adaptive tokens for the encoder.
Key results
- On the AGORA test set, the 644* model reaches MVE 63.3 mm with 42.0 ms inference and 133.1 GMACs, close to Multi-HMR (1288) at 61.1 mm and 231.7 ms, and AiOS at 57.5 mm.
- Runs at 24 FPS, about 5x faster than the top-performing methods, making it the best real-time model in the comparison.
- On 3DPW, after finetuning, the method reports MVE 73.7 mm versus 75.9 mm for Multi-HMR (896), and on CMU Panoptic it reaches average MPJPE 84.2 mm versus 105.7 mm for PSVT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use scale-adaptive tokens instead of uniform input tokens | SAT-HMR · AGORA test | inference time (ms): 405.2 -> 42.0 | Helped |
| Use scale-adaptive tokens instead of uniform input tokens | SAT-HMR · 3DPW test | MVE (mm): 75.9 -> 73.7 | Helped |
| Use scale-adaptive tokens instead of uniform input tokens | SAT-HMR · MuPoTS test | PCK All: 85.0 -> 89.0 | Helped |
| Discard all background tokens (ablation) | SAT-HMR · BEDLAM validation, Avg. | MVE (mm): 56.0 -> 57.2 | Hurt |
| Skip pooling background tokens (ablation) | SAT-HMR · BEDLAM validation, Avg. | MVE (mm): 56.0 -> 56.1 | No clear effect |
| Pool background tokens twice (ablation) | SAT-HMR · BEDLAM validation, Avg. | MVE (mm): 56.0 -> 56.3 | Hurt |
| Increase baseline input resolution from 518 to 1288 | DETR-style baseline · AGORA validation, individuals at 0-10% scale | MVE (mm): nearly 35mm | Helped |
| Increase baseline input resolution | DETR-style baseline · AGORA validation | FPS: 32 FPS to 5 FPS | Hurt |
Surprising findings
- Discarding all background tokens keeps performance stable for the smallest-scale individuals but degrades for larger scales, indicating background context matters more for large people.
Limitations
- Lacks height-aware scale estimation, which may cause depth errors.
- Only body mesh estimation; whole-body (SMPL-X hands/face) is not covered.
Experimental setup
Datasets: AGORA BEDLAM COCO CrowdPose MPII H3.6M 3DPW NeuralAnnot pseudo labels
Benchmarks: AGORA 3DPW MuPoTS-3D CMU Panoptic BEDLAM
Metrics: MVE MPJPE PA-MPJPE F1-score precision recall NMVE NMJE PCK inference time (ms)
Baselines: ROMP BEV PSVT AiOS Multi-HMR 3DCrowdNet CRMH
Builds on: DETR DINOv2 Multi-HMR AiOS TORE
Closest papers by meaning
- TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation (CVPR 2024 · similarity 0.87)
- GenHMR: Generative Human Mesh Recovery (AAAI 2025 · similarity 0.87)
- Reconstructing People, Places, and Cameras (CVPR 2025 · similarity 0.87)
- MEGA: Masked Generative Autoencoder for Human Mesh Recovery (CVPR 2025 · similarity 0.86)
- ADHMR: Aligning Diffusion-based Human Mesh Recovery via Direct Preference Optimization (ICML 2025 · similarity 0.86)
- AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.