Research map / 3D vision / Human pose estimation
M3GYM: A Large-Scale Multimodal Multi-view Multi-person Pose Dataset for Fitness Activity Understanding in Real-world Settings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces M3GYM, a large-scale multimodal, multi-view, multi-person human pose dataset from real gym sessions (82 sessions, 8 cameras, 47M frames) with 2D/3D keypoints, meshes, subject IDs, and action labels/assessments, and benchmarks pose estimation and mesh recovery methods.
Problem
Existing human pose datasets are either captured in custom laboratories, contain only single individuals, or focus on daily activities, lacking real-world multi-person fitness scenes with heavy occlusion, mirrors, and varied lighting.
Approach
A real-gym multi-view dataset of fitness, Pilates, and Yoga sessions with many subjects per scene, annotated via a semi-automated pipeline (multi-model 2D voting, triangulation, physics-based calibration, manual inspection, SMPLify meshes, and expert action labels/assessments).
Key results
- Fine-tuning ViTPose on M3GYM raises 2D AP from 66.3 to 73.1 (inference vs fine-tuned).
- Multi-view 3D MPJPE drops from 123.6 mm to 112.3 mm for ViTPose after fine-tuning on M3GYM.
- Cross-domain: a model trained on Yoga reaches 90.8 AP on Pilates but only 56.1 AP on Normal sessions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune ViTPose on M3GYM | ViTPose · M3GYM test | AP: 66.3 -> 73.1 | Helped |
| Fine-tune RTMPose on M3GYM | RTMPose · M3GYM test | AP: 64.0 -> 72.8 | Helped |
| Fine-tune RTMPose on M3GYM | RTMPose with MV-Pose reconstruction · M3GYM test | MPJPE (mm): 132.4 -> 113.5 | Helped |
| Fine-tune ViTPose on M3GYM | ViTPose with MV-Pose reconstruction · M3GYM test | MPJPE (mm): 123.6 -> 112.3 | Helped |
| Fine-tune VideoPose3D on M3GYM | VideoPose3D · M3GYM test | MPJPE (mm): 165.4 -> 126.4 | Helped |
| Fine-tune MotionBERT on M3GYM | MotionBERT · M3GYM test | MPJPE (mm): 158.9 -> 123.7 | Helped |
| Fine-tune VideoPose3D on M3GYM | VideoPose3D · M3GYM test | PA-MPJPE (mm): 115.1 -> 76.6 | Helped |
| Fine-tune MotionBERT on M3GYM | MotionBERT · M3GYM test | PA-MPJPE (mm): 111.2 -> 77.9 | Helped |
| Fine-tune OSX-SMPL on M3GYM | OSX-SMPL · M3GYM test | MPJPE (mm): 153.5 -> 123.8 | Helped |
| Fine-tune SMPLer-L on M3GYM | SMPLer-L · M3GYM test | PA-MPJPE (mm): 94.3 -> 67.2 | Helped |
| Train the 2D pose model on Yoga sessions rather than Normal sessions | 2D pose estimation model · Pilates | AP: 32.0 -> 90.8 | Helped |
| Train the 2D pose model on Yoga sessions rather than Pilates sessions | 2D pose estimation model · Normal | AP: 16.0 -> 56.1 | Helped |
Surprising findings
- Models trained on Yoga sessions generalize poorly to Normal sessions (AP 56.1) despite strong Pilates performance.
- Bottom-up methods DEKR and ED-Pose score very low on M3GYM inference (AP 19.9 and 14.0), while recall stays high.
Limitations
- Pilates and Yoga make up only 31 of 82 sessions and some lighting conditions are rare and unevenly distributed.
- Annotation relies on semi-automated pipeline with manual inspection; many frames are hard cases requiring adjustment.
- Mesh and 3D labels depend on triangulation and SMPLify fitting rather than direct capture.
Experimental setup
Datasets: M3GYM COCO17 BODY25
Benchmarks: M3GYM
Metrics: AP AP50 AP75 AR AR50 AR75 MPJPE PA-MPJPE
Baselines: DEKR YOLO-Pose ED-Pose DWPose RTMPose ViTPose SimpleBaseline VideoPose3D MotionBERT PyMAF OSX-SMPL SMPLer-L
Builds on: MV-Pose FreeMan HuMMan SMPLify
Closest papers by meaning
- Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectives (CVPR 2024 · similarity 0.87)
- PoseBH: Prototypical Multi-Dataset Training Beyond Human Pose Estimation (CVPR 2025 · similarity 0.85)
- EgoSim: An Egocentric Multi-view Simulator and Real Dataset for Body-worn Cameras during Motion and Activity (NeurIPS 2024 · similarity 0.85)
- Learning from Synthetic Human Group Activities (CVPR 2024 · similarity 0.85)
- HardMo: A Large-Scale Hardcase Dataset for Motion Capture (CVPR 2024 · similarity 0.84)
- Optimizing Human Pose Estimation Through Focused Human and Joint Regions (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.