Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Universal Actions for Enhanced Embodied Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces UniAct, a 0.5B embodied foundation model that learns a discrete universal action codebook from heterogeneous robot data via a VLM extractor with Gumbel-Softmax and lightweight embodiment-specific decoder heads.
Problem
Action-space heterogeneity across robots (different embodiments and control interfaces) prevents effective cross-domain data sharing and training of generalist embodied foundation models.
Approach
Learn a vector-quantized universal action codebook of embodiment-agnostic atomic behaviors, extracted by a fine-tuned VLM conditioned on observation and language goal (not on observation changes), then decode to each robot's actions with small per-embodiment MLP heads.
Key results
- UniAct-0.5B outperforms 14x larger OpenVLA-7B and LAPA-7B on visual, motion, and physical generalization on the WidowX real-world evaluation.
- UniAct-0.5B surpasses baselines across all LIBERO suites with a significant overall average improvement.
- On fast adaptation to the AIRBOT robot, UniAct fine-tunes only 4M of 500M parameters (0.8%) versus 1.4% for OpenVLA and 2% for Octo, while achieving consistently strong results across control types.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use UniAct-0.5B instead of LLaVa-OV-0.5B | Bimanual AIRBOT · Sweep | success rate: 7.5 -> 45 | Helped |
| Use UniAct-0.5B instead of LLaVa-OV-0.5B | Bimanual AIRBOT · Plate | success rate: 20 -> 62.5 | Helped |
| Use UniAct-0.5B instead of LLaVa-OV-0.5B | Bimanual AIRBOT · Fold Towel | success rate: 2.5 -> 50 | Helped |
| Use UniAct-0.5B instead of LLaVa-OV-0.5B | Bimanual AIRBOT · Cup on Plate | success rate: 15 -> 65 | Helped |
Surprising findings
- At least 40% of the 256 universal action codes decode to exactly consistent behaviors across different robots, even across sim-to-real gaps.
- A 0.5B model matches OpenVLA-7B on semantic generalization and language grounding.
Limitations
- Trained with only a 0.5B instantiation due to resource constraints.
- Evaluated mostly on single-arm robotic platforms.
- Only about 40% of universal action codes show exact cross-robot consistency.
Experimental setup
Datasets: Open-X Embodiment LIBERO DROID Bridge
Benchmarks: LIBERO WidowX real-world suite (19 tasks) AIRBOT fast adaptation
Metrics: success rate JS divergence of universal action utilization
Baselines: Octo CrossFormer OpenVLA LAPA VQ-BeT LLaVa-OV-0.5B
Builds on: LLaVA-OneVision VQ-VAE Gumbel-Softmax LAPA ACT
Closest papers by meaning
- Self-Improving Embodied Foundation Models (NeurIPS 2025 · similarity 0.89)
- Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models (NeurIPS 2026 · similarity 0.88)
- Motus: A Unified Latent Action World Model (CVPR 2026 · similarity 0.88)
- CrafterDojo: A Suite of Foundation Models for Building Open-Ended Embodied Agents in Crafter (NeurIPS 2026 · similarity 0.87)
- EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation (NeurIPS 2025 · similarity 0.87)
- Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.