Research map / Video / Video segmentation and tracking
FLD: Fourier Latent Dynamics for Structured Motion Representation and Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Fourier Latent Dynamics (FLD), a self-supervised periodic latent dynamics model extending the Periodic Autoencoder to predict motion, and uses it to parameterize physics-based humanoid motion tracking with a fallback mechanism for risky targets.
Problem
Motion trajectory data is sparse and does not cover the space of motions, so physics-based motion learning policies generalize poorly to unseen or interpolated motions and cannot safely handle risky tracking targets.
Approach
Extends the Periodic Autoencoder with multi-step latent dynamics: a constant frequency/amplitude/offset parameterization with a propagating latent phase, trained with a multi-step prediction loss. The prediction loss on online input also serves as an accept/reject test, with fallback to latent-propagated safe targets.
Key results
- FLD shows the lowest relative prediction error on an unseen diagonal run trajectory, with errors accumulating much more for FF and PAE baselines over long horizons.
- The fallback mechanism rejects an out-of-distribution spinkick target at t=200 and reverts to safe forward-stride references.
- Latent parameterization space interpolates smoothly between step-in-place and forward stride, giving smooth transitions in online tracking.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| enforce latent dynamics with forward propagation | FLD motion representation · Unseen-motion prediction | relative prediction error | Helped |
| enforce latent dynamics with forward propagation | FLD motion representation · Motion representation | temporal structure | Helped |
| replace FLD latent dynamics with local reconstruction only (N = 0) (ablation) | FLD · Unseen-motion prediction | relative prediction error | Hurt |
| use ALP-GMM curriculum sampling instead of fixed offline target sampling (ablation) | FLD motion-tracking policy · Motion tracking generality | tracking performance | Helped |
| use offline GMM sampling instead of ALP-GMM curriculum sampling (ablation) | FLD motion-tracking policy · Motion tracking generality | tracking performance | Mixed |
| use random latent-space sampling instead of ALP-GMM curriculum sampling (ablation) | FLD motion-tracking policy · Motion tracking generality | tracking performance | Hurt |
Limitations
- Relies on periodic or quasi-periodic motions; the quasi-constant parameterization assumption may fail for non-periodic motions.
- Depends on motion-capture data availability and accurate runtime input, as noted in the ethics statement.
- Acceptance threshold is derived from training statistics and the fallback relies on safe propagated targets.
Experimental setup
Datasets: human locomotion clips retargeted to MIT Humanoid
Benchmarks: motion reconstruction and prediction of unseen diagonal run trajectory online motion tracking with fallback on spinkick skill sampler comparison for tracking generality
Metrics: relative prediction error tracking reward normalized episodic tracking performance
Baselines: Periodic Autoencoder Variational Autoencoder feed-forward model offline point sampler offline GMM sampler random sampler
Builds on: Periodic Autoencoder Proximal Policy Optimization ALP-GMM
Closest papers by meaning
- ODEWorld: A Continuous Predictive Architecture via Physical-Time Flow (NeurIPS 2026 · similarity 0.85)
- Learning Physics From Video: Unsupervised Physical Parameter Estimation for Continuous Dynamical Systems (CVPR 2025 · similarity 0.84)
- HEIR: Learning Graph-Based Motion Hierarchies (NeurIPS 2025 · similarity 0.84)
- MotionMix: Weakly-Supervised Diffusion for Controllable Motion Generation (AAAI 2024 · similarity 0.84)
- Contrast encodes inductive bias: separating slow noise from dynamics in predictive representation learning (NeurIPS 2026 · similarity 0.84)
- Meta-Dynamical State Space Models for Integrative Neural Data Analysis (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.