Research map / Video / Action recognition
LAOF: Robust Latent Action Learning with Optical Flow Constraints
Xizhou Bu, Jiexi Lyu, Fulei Sun, Ruichen Yang, Zhiqiang Ma, Wei Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add optical flow constraint (flow decoder on latent action) to LAPO (ablation) | LAPO · LIBERO (4 suites) | success rate: avg +4.2 points over LAPO | Improved |
| add optical flow constraints to action-supervised LAOM (ablation) | LAOM-Action · LIBERO (1% labels) | success rate: avg +8.7 -> +11.5 over LAPO | Improved |
| add optical flow constraint to LAPO (ablation) | LAPO · PROCGEN (4 games) | normalized return: avg +0.16 over LAPO | Improved |
| add optical flow constraints to LAOM-Action (ablation) | LAOM-Action · PROCGEN (1% labels) | normalized return: avg +0.07 -> +0.22 | Improved |
| replace IDM future-frame input with frame differences (CoMo) (ablation) | LAPO · LIBERO SPATIAL and GOAL | success rate: SPATIAL 80.4 -> 74.1; GOAL 84.0 -> 80.8 | Hurt |
| add optical flow to CoMo (ablation) | CoMo · LIBERO | avg success rate: +0.5 -> +4.0 | Improved |
| remove forward dynamics model (flow-only z_t) (ablation) | LAOF · LIBERO and PROCGEN | avg improvement vs LAPO: +4.2 -> +3.6; +16.20 -> +13.2 | Hurt |
| integrate flow decoder into FDM instead of dedicated decoder (ablation) | LAOF · LIBERO and PROCGEN | avg improvement vs LAPO: +4.2 -> +3.3; +0.16 -> +0.13 | Hurt |
| feed current state to flow decoder along with latent action (ablation) | LAOF-Only · LIBERO and PROCGEN | avg improvement vs LAPO: +3.6 -> +0.8 | Hurt |
| increase action label ratio to 100% (ablation) | optical flow constraints · PROCGEN BIGFISH and CHASER | action accuracy | Hurt |
| use continuous instead of discrete latent actions | latent action form · LIBERO and PROCGEN downstream | success rate / return | Improved |
| LAOF without action labels vs action-supervised LAOM-Action at 1% labels | LAOF · PROCGEN | action accuracy | Improved |
Closest papers by meaning
- LAFP: Preserving Latent Action Structure in Latent Policy Learning via Flow Matching (NeurIPS 2026 · similarity 0.90)
- Latent Action Learning Requires Supervision in the Presence of Distractors (ICML 2025 · similarity 0.89)
- Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representations (NeurIPS 2025 · similarity 0.88)
- Latent Action Pretraining from Videos (ICLR 2025 · similarity 0.87)
- PlaySlot: Learning Inverse Latent Dynamics for Controllable Object-Centric Video Prediction and Planning (ICML 2025 · similarity 0.86)
- RotVLA: Rotational Latent Action for Vision-Language-Action Model (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
