Research map / 3D vision / Human pose estimation
RTMO: Towards High-Performance One-Stage Real-Time Multi-Person Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RTMO, a one-stage YOLO-based multi-person pose estimator that integrates coordinate classification via dynamic bin allocation/encoding and an MLE loss, achieving top-down-level accuracy at real-time speed.
Problem
Real-time multi-person pose estimation must balance speed and accuracy; top-down methods slow with more people, while existing one-stage methods lag in accuracy because they directly regress keypoint coordinates.
Approach
Brings coordinate classification (dual 1-D heatmaps) into a YOLO-style one-stage detector by localizing bins to each predicted bounding box (Dynamic Coordinate Classifier with sine positional-encoding bin representations) and trains with a learnable-variance Laplace MLE loss instead of KL divergence.
Key results
- RTMO-l reaches 74.8% AP on COCO val2017 (with additional data) and 141 FPS on a single V100 GPU.
- RTMO outperforms state-of-the-art one-stage methods by 1.1% AP on COCO and runs about 9x faster with the same backbone.
- On CrowdPose, RTMO-l achieves 73.2% AP without extra data, a new one-stage state of the art, and 83.8% AP with additional data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use coordinate classification with DCC and MLE instead of regression (ablation) | RTMO-s · COCO val2017 | AP: 65.6 -> 67.6 | Helped |
| use coordinate classification with DCC and MLE instead of regression (ablation) | RTMO-s · CrowdPose | AP: 66.1 -> 67.2 | Helped |
| use MLE instead of KLD loss (ablation) | RTMO-s coordinate classification · COCO val2017 | AP: 64.4 -> 67.6 | Helped |
| use MLE instead of KLD loss (ablation) | RTMO-s coordinate classification · CrowdPose | AP: 62.5 -> 67.2 | Helped |
| add dynamic bin allocation without dynamic bin encoding (ablation) | RTMO-s coordinate classification with MLE · COCO val2017 | AP: 66.7 -> 65.6 | Hurt |
| add dynamic bin allocation without dynamic bin encoding (ablation) | RTMO-s coordinate classification with MLE · CrowdPose | AP: 65.8 -> 65.2 | Hurt |
| add dynamic bin encoding to dynamic bin allocation (ablation) | RTMO-s coordinate classification with MLE · COCO val2017 | AP: 65.6 -> 67.6 | Helped |
| add dynamic bin encoding to dynamic bin allocation (ablation) | RTMO-s coordinate classification with MLE · CrowdPose | AP: 65.2 -> 67.2 | Helped |
| omit P3 features (ablation) | RTMO-s · COCO val2017 | AP: 67.6 -> 67.6 | No clear effect |
| omit P3 features (ablation) | RTMO-m · COCO val2017 | AP: 71.4 -> 71.2 | Hurt |
| omit P3 features (ablation) | RTMO-l · COCO val2017 | AP: 72.7 -> 72.5 | Hurt |
Surprising findings
- Learnable variance in KLD-based coordinate classification is problematic: the model tends to predict large variance, so MLE loss is needed.
- Dropping the P3 feature map cut head FLOPs substantially (P3 was 78.5% of head FLOPs) with minimal accuracy loss.
Limitations
- Top-down methods remain competitive in some regimes; RTMO-l without extra data trails ED-Pose with Swin-L on COCO test-dev.
- Best results require training on additional datasets beyond COCO train2017.
Experimental setup
Datasets: COCO train2017 CrowdPose AIC MPII JHMDB Halpe PoseTrack18
Benchmarks: COCO val2017 COCO test-dev CrowdPose
Metrics: AP AR inference latency (ms) FPS
Baselines: YOLOX-Pose YOLOv8-Pose KAPAO RTMPose ED-Pose PETR CID DEKR RLE SimCC
Builds on: SimCC RTMPose YOLO-Pose SimOTA DFL
Closest papers by meaning
- MotionMap: Representing Multimodality in Human Pose Forecasting (CVPR 2025 · similarity 0.87)
- FCPose: Fully Convolutional Multi-Person Pose Estimation With Dynamic Instance-Aware Convolutions (CVPR 2021 · similarity 0.86)
- MV-SSM: Multi-View State Space Modeling for 3D Human Pose Estimation (CVPR 2025 · similarity 0.85)
- PoseMamba: Monocular 3D Human Pose Estimation with Bidirectional Global-Local Spatio-Temporal State Space Model (AAAI 2025 · similarity 0.84)
- Co-op: Correspondence-based Novel Object Pose Estimation (CVPR 2025 · similarity 0.84)
- PoseIRM: Enhance 3D Human Pose Estimation on Unseen Camera Settings via Invariant Risk Minimization (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.