Research map / 3D vision / Human pose estimation
LAL: Enhancing 3D Human Motion Prediction with Latency-aware Auxiliary Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LAL, a latency-aware auxiliary learning framework that uses observed motion during the robot's response latency as auxiliary input to improve 3D human motion prediction of the valid future frames, with no extra test-time cost.
Problem
Existing 3D human motion prediction methods assume instantaneous robot reaction, so predictions within the response-latency window are outdated and useless for collaboration; naively dropping that window does not help.
Approach
Two branches: a primary branch predicts only the valid future from the observation, and an auxiliary branch predicts the same target from observation plus latency-period motion; separate encoders are aligned by a feature consistency loss (instead of shared encoder parameters), and an optional feature-statistics (mean/covariance) alignment refines the primary branch.
Key results
- On Human3.6M, STS-LAL-3 reduces average error at 1000 ms (STS-LAL-5 reaches 106.6 vs. 113.3 for STS at 1000 ms in the TL-based table; per-setting gains reported for LTD, STS, SPGSN).
- SPGSN-LAL-3 improves SPGSN's 400 ms error on Human3.6M smoking/directions/sitting/waiting, e.g., 31.9 vs 34.6 (smoking, 400 ms).
- On CMU-Mocap at 1000 ms, SPGSN-LAL-3 gives 75.1 vs 77.8 for SPGSN, and LAL keeps testing time unchanged (36.76 vs 36.87 s running time for LTD).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add latency-aware auxiliary learning (TL=3) to LTD | LTD · Human3.6M average, 400 ms | MPJPE: 63.5 -> 59.6 | Helped |
| Add latency-aware auxiliary learning (TL=5) to LTD | LTD · Human3.6M average, 1000 ms | MPJPE: 114.3 -> 108.5 | Helped |
| Add latency-aware auxiliary learning (TL=3) to STS | STS · Human3.6M average, 400 ms | MPJPE: 60.6 -> 51.4 | Helped |
| Add latency-aware auxiliary learning (TL=5) to SPGSN | SPGSN · Human3.6M average, 1000 ms | MPJPE: 109.6 -> 104.1 | Helped |
| Add latency-aware auxiliary learning (TL=3) to LTD | LTD · CMU-Mocap, 400 ms | MPJPE: 41.0 -> 38.5 | Helped |
| Add latency-aware auxiliary learning (TL=5) to SPGSN | SPGSN · CMU-Mocap, 1000 ms | MPJPE: 77.8 -> 74.2 | Helped |
| Increase latency length from TL=3 to TL=5 (ablation) | LAL-assisted LTD · Human3.6M average, 1000 ms | MPJPE: 111.7 -> 108.5 | Helped |
| Increase latency length from TL=3 to TL=5 (ablation) | LAL-assisted STS · Human3.6M average, 400 ms | MPJPE: 51.4 -> 49.2 | Helped |
| Increase latency length from TL=3 to TL=5 (ablation) | LAL-assisted SPGSN · CMU-Mocap, 1000 ms | MPJPE: 75.1 -> 74.2 | Helped |
| Add feature-statistics alignment refinement (ablation) | LAL-assisted motion prediction · Human3.6M | MPJPE: 63.5 -> 58.4; 114.3 -> 107.4 | Helped |
| Discard latency-region supervision and train only on valid targets (ablation) | STS with TL=3 · Human3.6M | MPJPE | No clear effect |
| Reduce latency-region loss weight to θ=0.2, 0.5, or 0.8 (ablation) | STS with TL=3 · Human3.6M | MPJPE | No clear effect |
Surprising findings
- Simply removing the latency-period supervision (training only on the valid target) gives no improvement over the original baseline.
- Sharing an encoder with a linear or temporal-convolution dimension-alignment layer yields only limited gains, whereas the separate-encoder consistency loss works better.
Limitations
- Assumes encoder-decoder architectures; the approach only suits mainstream encoder-decoder motion predictors.
- Latency length is fixed per model (TL) and the authors do not increase TL beyond 5 poses due to efficiency concerns.
Experimental setup
Datasets: Human3.6M CMU-Mocap 3DPW
Benchmarks: Human3.6M CMU-Mocap 3DPW
Metrics: MPJPE
Baselines: Res. sup DMGNN PGBIG LTD STS SPGSN
Builds on: LTD STS SPGSN
Closest papers by meaning
- ALIEN: Implicit Neural Representations for Human Motion Prediction under Arbitrary Latency (CVPR 2025 · similarity 0.87)
- SimpliHuMoN: Simplifying Human Motion Prediction (NeurIPS 2026 · similarity 0.86)
- Multimodal Sense-Informed Forecasting of 3D Human Motions (CVPR 2024 · similarity 0.86)
- MoML: Online Meta Adaptation for 3D Human Motion Prediction (CVPR 2024 · similarity 0.85)
- Harmonizing Stochasticity and Determinism: Scene-responsive Diverse Human Motion Prediction (NeurIPS 2024 · similarity 0.85)
- Human Motion Prediction Under Unexpected Perturbation (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.