Research map / 3D vision / Human pose estimation
CoMotion: Concurrent Multi-person 3D Motion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CoMotion, an online recurrent model that detects and tracks multi-person 3D SMPL poses from monocular video by updating tracks directly from image features rather than per-frame association.
Problem
Online multi-person 3D pose estimation and tracking from monocular video, especially through occlusion in crowded scenes, where tracking-by-detection methods are fragile and slow.
Approach
Tracking-by-attention for 3D poses: a single-shot detection module proposes new tracks while a recurrent (GRU) pose update module cross-attends to image features to refine all existing tracks each frame, including invisible ones, with heuristic track management based on a modified OKS.
Key results
- On PoseTrack21, CoMotion reaches 67.6 MOTA and 77.9 IDF1 vs. 59.5 and 69.3 for the best prior method (Tracktor++); with the corrected ignore-region evaluation, it reaches 71.4 MOTA vs. 56.7 for 4D Humans.
- CoMotion runs at about 176 ms per frame on PoseTrack21 validation, roughly 12x faster than 4D Humans (2163 ms) and 1.4x faster than PARE (258 ms).
- On 3DPW oracle-crop, CoMotion achieves 63.6 MPJPE and 36.1 PA-MPJPE, the best PA-MPJPE in the table.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CoMotion instead of Tracktor++ | PoseTrack21 tracking · PoseTrack21 | MOTA: 59.5 -> 67.6 | Helped |
| Use CoMotion instead of Tracktor++ | PoseTrack21 tracking · PoseTrack21 | IDF1: 69.3 -> 77.9 | Helped |
| Add recurrent GRU update to the hidden state (ablation) | CoMotion · PoseTrack | 2D@0.05: 73.0 -> 74.9 | Helped |
| Add recurrent GRU update to the hidden state (ablation) | CoMotion · PoseTrack | IDF1: 76.9 -> 78.6 | Helped |
| Add a learned hidden state (ablation) | CoMotion without hidden state · PoseTrack | 2D@0.05: 72.6 -> 74.9 | Helped |
| Add a learned hidden state (ablation) | CoMotion without hidden state · PoseTrack | IDs: 559 -> 491 | Helped |
| Add PoseTrack pseudo-labeled video training data (ablation) | CoMotion trained without PoseTrack data · PoseTrack | MOTA: 69.8 -> 70.9 | Helped |
| Add DanceTrack pseudo-labeled video training data (ablation) | CoMotion trained without DanceTrack data · PoseTrack | MOTA: 68.4 -> 70.9 | Helped |
| Add DanceTrack pseudo-labeled video training data (ablation) | CoMotion trained without DanceTrack data · PoseTrack | IDs: 576 -> 491 | Helped |
| Add PoseTrack pseudo-labeled video training data (ablation) | CoMotion trained without PoseTrack data · PoseTrack | IDs: 703 -> 491 | Helped |
| Fine-tune on longer sequences in Stage 3 (ablation) | CoMotion after Stage 2 · PoseTrack | MOTA: 70.9 -> 71.4 | Helped |
| Fine-tune on longer sequences in Stage 3 (ablation) | CoMotion after Stage 2 · PoseTrack | IDs: 491 -> 382 | Helped |
Surprising findings
- Full-image pose estimation without an oracle bounding box is only modestly worse than oracle-crop performance (e.g., 3DPW MPJPE 60.0 full vs. 63.6 oracle crop).
- A bug in the PoseTrack21 evaluation code penalized predictions inside ignore regions as false positives; fixing it changes CoMotion's MOTA from 67.6 to 71.4 and 4D Humans' from 56.7 to 56.7 on the corrected evaluation.
Limitations
- Tracks can collapse into a single identity and identity switches occur; the model lacks physical grounding.
- Track management relies on hand-tuned heuristics, creating a train-test discrepancy with fixed short training clips.
- Pseudo-labels from NLF are unreliable for close proximity people and close-ups, so such samples are ignored in training; no absolute scale or camera motion modeling.
Experimental setup
Datasets: InstaVariety COCO MPII PoseTrack DanceTrack BEDLAM WHAC-A-MOLE AMASS EgoHumans
Benchmarks: PoseTrack21 PoseTrack18 COCO 3DPW EgoHumans
Metrics: MOTA IDF1 IDP IDR HOTA IDs PCKn@0.05 PCKn@0.1 MPJPE PA-MPJPE
Baselines: 4D Humans PHALP TRMOT FairMOT CorrTrack Tracktor++ PARE PyMAF CLIFF PyMAF-X HMR 2.0
Builds on: 4D Humans SMPL ConvNeXtV2 tracking by attention NLF
Closest papers by meaning
- ArtTrack: Articulated Multi-Person Tracking in the Wild (CVPR 2017 · similarity 0.90)
- PoseTrack: Joint Multi-Person Pose Estimation and Tracking (CVPR 2017 · similarity 0.89)
- TesseTrack: End-to-End Learnable Multi-Person Articulated 3D Pose Tracking (CVPR 2021 · similarity 0.89)
- PoseTrack21: A Dataset for Person Search, Multi-Object Tracking and Multi-Person Pose Tracking (CVPR 2022 · similarity 0.89)
- Monocular 3D Pose and Shape Estimation of Multiple People in Natural Scenes - The Importance of Multiple Scene Constraints (CVPR 2018 · similarity 0.89)
- Motions as Queries: One-Stage Multi-Person Holistic Human Motion Capture (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.