Research map / Video / Action recognition
H-MoRe: Learning Human-centric Motion Representation for Action Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes H-MoRe, a self-supervised human-centric optical-flow-like motion representation (world-local flows) learned from real-world video with skeleton and boundary constraints, and shows gains in gait recognition, action recognition, and video generation.
Problem
Optical flow treats background and foreground motion alike and lacks body shape, while pose lacks shape; existing flow learning relies on synthetic supervised data, hurting human-centric action analysis.
Approach
Learn a flow estimator from real human videos without ground truth using a joint constraint: a skeleton constraint (kinematic consistency of body points with 2D pose offsets) and a boundary constraint (Chamfer alignment of flow edges with human silhouettes). Adds world flow (relative to environment) and local flow (relative to subject, derived via vector decomposition with a lightweight network) a
Key results
- On CASIA-B gait recognition with GaitBase, H-MoRe reaches 87.66% CL@R1 versus 78.02% without flow, a 9.64-point gain.
- On Diving48 action recognition, H-MoRe improves Acc@1 to 72.99% versus 64.07% without flow (8.92-point gain) and reaches Acc@5 97.62%.
- On UTD-MHAD video generation with LGC-VD, H-MoRe lowers FVD to 108.38 from 118.13 with RAFT (5.74% reduction) and raises SSIM to 0.9574.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use H-MoRe instead of optical flow | GaitBase · CASIA-B, CL | R1: 86.12 -> 87.66 | Helped |
| use H-MoRe instead of optical flow | Video-FocalNets · Diving48 | Acc@1: 64.07 -> 72.99 | Helped |
| use H-MoRe instead of optical flow | LGC-VD · UTD-MHAD | FVD: 114.98 -> 108.38 | Helped |
| add skeleton constraint F (ablation) | H-MoRe · Diving48 | Acc@1: 68.17 -> 72.99 | Helped |
| add boundary constraint G (ablation) | H-MoRe · Diving48 | Acc@1: 72.13 -> 72.99 | Helped |
| add both skeleton constraint F and boundary constraint G (ablation) | H-MoRe · Diving48 | Acc@5: 95.23 -> 97.62 | Helped |
| add both skeleton constraint F and boundary constraint G (ablation) | H-MoRe · CASIA-B, NM | R1: 96.99 -> 98.26 | Helped |
| add local flow Ml to world flow Mw (ablation) | H-MoRe · Diving48 | Acc@1: 70.91 -> 72.99 | Helped |
| add world flow Mw to local flow Ml (ablation) | H-MoRe · Diving48 | Acc@1: 72.64 -> 72.99 | Helped |
Surprising findings
- FlowFormer++ underperforms the no-flow baseline on Diving48 action recognition (64.28 vs 64.07 Acc@1 comparable, and lower than most flow methods).
- Local flow alone outperforms world flow alone on gait and action recognition, suggesting subject-relative motion is more useful for these tasks.
Limitations
- Validated only in 2D with a limited number of subjects.
- Multi-subject overlap robustness tested only on Diving48 with bounded overlap bins.
Experimental setup
Datasets: CASIA-B Diving48 UTD-MHAD
Benchmarks: CASIA-B Diving48 UTD-MHAD
Metrics: Rank-1 accuracy (NM, BG, CL) Acc@1 Acc@5 SSIM FVD FLOPs Params
Baselines: RAFT GMA GMFlow CRAFT SKFlow VideoFlow FlowFormer++ MHOF 2D Pose (AlphaPose) 3D Pose (MotionBERT)
Builds on: RAFT ED-Pose U2Net LGC-VD GaitBase
Closest papers by meaning
- H-Flow: Self-supervised Human Scene Flow via Physics-inspired Joint Multi-modal Learning (NeurIPS 2026 · similarity 0.90)
- PhysPT: Physics-aware Pretrained Transformer for Estimating Human Dynamics from Monocular Videos (CVPR 2024 · similarity 0.86)
- Scaling Large Motion Models with Million-Level Human Motions (ICML 2025 · similarity 0.86)
- HVIS: A Human-like Vision and Inference System for Human Motion Prediction (AAAI 2025 · similarity 0.85)
- Motions as Queries: One-Stage Multi-Person Holistic Human Motion Capture (CVPR 2025 · similarity 0.85)
- SimpliHuMoN: Simplifying Human Motion Prediction (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.