Research map / 3D vision / Human pose estimation

Neural-Behavioral Representation of Natural Whole-body Movement in Monkeys

NeurIPS 2026 · 3D vision › Human pose estimation · approach: Physics and sensor-based motion capture · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds a neural-behavioral recording platform (62-channel epidural ECoG plus 8-camera 3D motion capture) for one freely moving monkey and an autoregressive VAE-style mixture-of-experts model that reconstructs whole-body kinematics from neural signals and a learned behavior prior.

Problem

Decoding continuous natural whole-body movement of freely moving primates from large-scale cortical activity, which prior motor decoding work (constrained tasks, few limbs) has not addressed.

Approach

A conditional autoregressive encoder-decoder (VAE with mixture-of-experts decoder) that combines a latent behavior prior with epidural neural features and the previous predicted pose, generating whole-body kinematics without explicit physical constraints.

Key results

• Neural-Behavioral Model achieved horizontal ADE 6.97 cm / FDE 14.94 cm / r 0.47 and vertical ADE 6.18 cm / FDE 13.56 cm / r 0.40 over 3 s, versus Behavioral Model (ADE 10.54 / 10.56) and LSTM (ADE 16.93 / 12.39).
• Average whole-body joint-position Pearson correlation of about 0.45 across natural movements.
• Neural-Behavioral Model outperformed the Behavioral Model beyond roughly 1.5 s of prediction horizon; at short horizons differences were not consistently significant.

Surprising findings

• Beta/gamma band coupling with movement was less consistent in free movement than in constrained tasks, with much spontaneous neural activity unrelated to movement.
• The LSTM baseline could not predict motion direction, so its results used ground-truth velocity direction combined with predicted magnitude.

Limitations

• Single monkey, single session used for modeling (30 min), limiting generalization claims.
• Errors accumulate in autoregressive rollout after about 3 s; struggles with fine-grained movements of subsets of joints.
• Neural signals not directly reconstructed; no neural reconstruction loss used.

Experimental setup

Datasets: in-house epidural neural and 8-camera motion capture data from one monkey (Macaca fascicularis)

Benchmarks: in-house freely moving monkey test split

Metrics: ADE FDE Pearson correlation r paired Wilcoxon signed-rank test with FDR correction

Baselines: Behavioral Model (no neural input) LSTM neural decoder

Builds on: ELBO / variational autoencoder DeepLabCut Anipose LSTM neural decoder

Closest papers by meaning

• SPINT: Spatial Permutation-Invariant Neural Transformer for Consistent Intracortical Motor Decoding (NeurIPS 2025 · similarity 0.86)
• Neural Embeddings Rank: Aligning 3D latent dynamics with movements (NeurIPS 2024 · similarity 0.85)
• Few-shot Algorithms for Consistent Neural Decoding (FALCON) Benchmark (NeurIPS 2024 · similarity 0.84)
• MonkeySee: Space-time-resolved reconstructions of natural images from macaque multi-unit activity (NeurIPS 2024 · similarity 0.84)
• Exploring the trade-off between deep-learning and explainable models for brain-machine interfaces (NeurIPS 2024 · similarity 0.84)
• Neural Encoding and Decoding at Scale (ICML 2025 · similarity 0.84)