Research map / Video / Video segmentation and tracking
HVIS: A Human-like Vision and Inference System for Human Motion Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HVIS, a human-like vision encoder (retina-analog and visual-cortex-analog components) plus a two-stage GAN-based inference module (spontaneous adversarial learning and deliberate training on hard joints) for 3D human motion prediction, reporting SOTA on H3.6M, CMU Mocap and G3D.
Problem
Predicting future 3D human poses from observed skeleton sequences is hard due to stochastic motion, high pose dimensionality, and difficulty capturing long-term dependencies, with easy-to-train joints misleading training.
Approach
Separates spatial and temporal graph encoding (retina-analog static/dynamic adjacency), encodes multi-scale joint/limb/pose features hierarchically (visual-cortex analog), and trains with a WGAN-style joint-level adversarial generator followed by a deliberate stage that re-trains only the hardest joints selected by a ranking of joint errors.
Key results
- Reports SOTA MPJPE on H3.6M, e.g., Ours 6.3 vs FDU 6.6 at 80ms for Directions and 59.8 vs 97.2 at 1000ms.
- Claims 19.8% improvement on Human3.6M, 15.7% on CMU Mocap and 11.1% on G3D over prior methods (abstract).
- Ablation on H3.6M: removing HVM raises error at 1000ms from 94.1 to 121.9.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HVIS instead of FDU | Human motion prediction · H3.6M, Purchase, 80 ms | MPJPE: 11.8 -> 11.0 | Helped |
| Use HVIS instead of FDU | Human motion prediction · H3.6M, Purchase, 1,000 ms | MPJPE: 130.7 -> 105.8 | Helped |
| Use HVIS instead of MSR-GCN | Human motion prediction · CMU, Basketball, 80 ms | MPJPE: 13.1 -> 10.5 | Helped |
| Use HVIS instead of MSR-GCN | Human motion prediction · CMU, Basketball, 1,000 ms | MPJPE: 97.7 -> 89.3 | Helped |
| Add HVM and HIM-TRN (ablation) | HVIS · H3.6M | MPJPE: 16.8 -> 7.8; 121.9 -> 94.1 | Helped |
| Add HVM (ablation) | HVIS · H3.6M | MPJPE: 10.8 -> 7.8; 101.0 -> 94.1 | Helped |
| Add HIM-TRN (ablation) | HVIS · H3.6M | MPJPE: 8.1 -> 7.8; 98.7 -> 94.1 | Helped |
| Add HIM-DLN (ablation) | HVIS · H3.6M | MPJPE: 8.1 -> 7.8; 98.7 -> 94.1 | Helped |
Limitations
- Each dataset and action is trained independently, limiting generality
- Transformer was reported to perform inconsistently and was not used; justification deferred to an unseen experimental section
- Several design choices (GRU over TCN in LTF) are justified by unreported evidence in an 'open-source library'
Experimental setup
Datasets: Human3.6M CMU Mocap G3D
Benchmarks: Human3.6M CMU Mocap G3D
Metrics: MPJPE
Baselines: res-GRU HPGAN BiGAN HMR LTD DMGNN HRI MSR-GCN SPGSN FDU
Builds on: WGAN Graph convolutional networks LTD MSR-GCN
Closest papers by meaning
- SimpliHuMoN: Simplifying Human Motion Prediction (NeurIPS 2026 · similarity 0.86)
- HMVLM:Human Motion-Vision-Language Model via MoE LoRA (NeurIPS 2025 · similarity 0.85)
- H-MoRe: Learning Human-centric Motion Representation for Action Analysis (CVPR 2025 · similarity 0.85)
- Physical Plausibility-aware Trajectory Prediction via Locomotion Embodiment (CVPR 2025 · similarity 0.85)
- AMD: Autoregressive Motion Diffusion (AAAI 2024 · similarity 0.85)
- FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observations (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.