Research map / Video / Action recognition
Stochastic Human Motion Prediction with Memory of Action Transition and Action Characteristic
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes two memory banks (Soft-transition Action Bank and Action Characteristic Bank) plus an Adaptive Attention Adjustment fusion strategy on a CVAE to improve action-driven stochastic human motion prediction.
Problem
Action-driven stochastic human motion prediction must generate smooth transitions between actions with varying speeds and capture action characteristics that are hard to learn when actions are similar.
Approach
Memory banks store action-transition features (queried with top-k soft action recognition from a GRU classifier) and per-action characteristic features; an adaptive weight alpha, driven by a running mean of the recognizer's cross-entropy loss over time, fuses the two retrieved features.
Key results
- Achieves Acc 95.23% vs WAT 92.6% on GRAB and 80.50% vs 76.0% on NTU RGB-D.
- On HumanAct12, Acc 61.57% vs WAT 59.0%, FID test 137.28 vs 164.38 (lower is better).
- Ablation: removing STAB drops GRAB accuracy from 95.23 to 92.18; top-2 soft search is best among k=1 to 4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed method instead of WAT | action-driven stochastic human motion prediction · GRAB | Acc: 92.6±0.6 -> 95.23±0.32 | Helped |
| Use the proposed method instead of WAT | action-driven stochastic human motion prediction · NTU | Acc: 76.0±0.2 -> 80.50±0.52 | Helped |
| Use the proposed method instead of WAT | action-driven stochastic human motion prediction · BABEL | Acc: 49.6±0.4 -> 55.37±0.38 | Helped |
| Use the proposed method instead of WAT | action-driven stochastic human motion prediction · HumanAct12 | Acc: 59.0±0.1 -> 61.57±0.32 | Helped |
| Add AAA feature fusion instead of mean query features (ablation) | proposed method · GRAB | Acc: 91.93±0.49 -> 95.23±0.32 | Helped |
| Add STAB (ablation) | proposed method · GRAB | Acc: 92.18±0.60 -> 95.23±0.32 | Helped |
| Add ACB (ablation) | proposed method · GRAB | Acc: 93.45±0.50 -> 95.23±0.32 | Helped |
| Use a running mean when updating α (ablation) | proposed method · GRAB | Acc: 90.84±0.90 -> 95.23±0.32 | Helped |
| Use top-1 rather than top-2 STAB categories (ablation) | proposed method · GRAB | Acc: 95.23±0.32 -> 93.03±0.75 | Hurt |
| Use top-3 rather than top-2 STAB categories (ablation) | proposed method · GRAB | Acc: 95.23±0.32 -> 93.74±0.59; Divw 1.14±0.04 -> 1.21±0.03 | Mixed |
| Use top-4 rather than top-2 STAB categories (ablation) | proposed method · GRAB | Acc: 95.23±0.32 -> 92.57±0.54; Divw 1.14±0.04 -> 1.22±0.03 | Mixed |
Surprising findings
- Larger k in soft search (k=3, 4) increases diversity but reduces accuracy, showing a diversity-quality trade-off.
Limitations
- Added memory and computation overhead (FPS 2.76 to 1.98; GPU memory 2262 to 2837 MB).
- Gains on NTU and BABEL are modest, and Div on NTU is slightly lower than WAT.
Experimental setup
Datasets: GRAB NTU RGB-D BABEL HumanAct12
Benchmarks: GRAB NTU RGB-D BABEL HumanAct12
Metrics: Acc FID Div Div_w (DTW-aligned diversity)
Baselines: Act2Mot DLow ACTOR WAT
Builds on: WAT CVAE DLow
Closest papers by meaning
- Weakly-Supervised Action Transition Learning for Stochastic Human Motion Prediction (CVPR 2022 · similarity 0.92)
- Action-guided 3D Human Motion Prediction (NeurIPS 2021 · similarity 0.89)
- MANTA: Diffusion Mamba for Efficient and Effective Stochastic Long-Term Dense Action Anticipation (CVPR 2025 · similarity 0.85)
- SimpliHuMoN: Simplifying Human Motion Prediction (NeurIPS 2026 · similarity 0.84)
- Visual Representation Learning with Stochastic Frame Prediction (ICML 2024 · similarity 0.84)
- SingularTrajectory: Universal Trajectory Predictor Using Diffusion Model (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.