Research map / Architectures and efficiency / Attention and transformers
BEAST: Efficient Tokenization of B-Splines Encoded Action Sequences for Imitation Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BEAST, a B-spline control-point action tokenizer that needs no tokenizer training and yields fixed-length tokens for parallel decoding, integrated into Florence-2, a decoder-only transformer, and ACT.
Problem
Action sequence tokenization for imitation learning: existing tokenizers (binning, VQ, BPE/FAST) either need separate tokenizer training or produce variable-length tokens, hindering parallel decoding, and ignore smoothness across action chunk boundaries.
Approach
Encode each normalized action chunk as least-squares/ridge-regressed clamped B-spline control points, quantize them to 256 bins (or keep them continuous), and flatten them into fixed-length tokens; clamped knots pin the first control point to the previous chunk's last action for smooth chunk transitions.
Key results
- BEAST-F reaches 4.42 average length on CALVIN ABC->D and 4.61 on ABCD->D, the best among the listed methods.
- BEAST-F achieves 617.3 Hz throughput and 19 ms latency on an RTX 4090, 2.14x faster than pi0 and 101.4x faster than OpenVLA.
- On LIBERO-Long BEAST-F reaches 86.4% success, the best among compared methods, while averaging 92.5% across suites versus pi0's 94.2%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace single-step or chunk-level binning with B-spline action tokens | decoder-only Transformer · Toy spline prediction | MSE: 0.0215±0.0216 -> 0.0004±0.0005; 0.0009±0.0013 -> 0.0004±0.0005 | Helped |
| use BEAST-F instead of VPP | CALVIN ABC→D · CALVIN ABC→D | average rollout length: 4.29 -> 4.42 | Helped |
| use BEAST-F instead of MDT | CALVIN ABCD→D · CALVIN ABCD→D | average rollout length: 4.52 -> 4.61 | Helped |
| use BEAST-F instead of π₀ | Florence-2 VLA · LIBERO Long | success rate: 85.2% -> 86.4 % | Helped |
| use BEAST-F instead of π₀ | Florence-2 VLA · LIBERO Average | success rate: 94.2% -> 92.5% | Hurt |
| use parallel decoding with BEAST-F instead of π₀ | Florence-2 VLA · Inference efficiency | throughput: 288.11 -> 617.3 | Helped |
| use parallel decoding with BEAST-F instead of π₀ | Florence-2 VLA · Inference efficiency | latency: 0.104 -> 0.019 | Helped |
| replace BEAST with Binning-F | Florence-2 VLA · CALVIN ABC average sequence length | average sequence length: 1.41 -> 4.43 | Hurt |
| use continuous tokens instead of discrete tokens (ablation) | BEAST-F · CALVIN ABC | task performance: 12.7% | Hurt |
| reduce basis functions from 10 to 5 (ablation) | BEAST-F · CALVIN ABC | average sequence length: 4.43 -> 3.88 | Hurt |
| increase basis functions from 10 to 15, 20, or 25 (ablation) | BEAST-F · CALVIN ABC | average sequence length: 4.43 -> 4.20; 4.43 -> 4.32; 4.43 -> 4.23 | Mixed |
| use Florence-2-base instead of Florence-2-large (ablation) | BEAST-F · CALVIN ABC | average sequence length: 3.98 -> 4.43; 11.3% | Hurt |
Surprising findings
- The smaller BEAST-D (76.57%) outperforms the Florence-based BEAST-F (52.86%) and pi0 (53.43%) on the real-world tasks, which the authors attribute to the small real-world dataset.
- Continuous BEAST tokens (BEAST-CT) underperform discrete tokens by 12.7%.
Limitations
- Performance is sensitive to the number of B-spline basis functions, which depends on trajectory smoothness and sampling frequency.
- Heuristic for choosing basis count is recommended rather than automatically optimized.
Experimental setup
Datasets: CALVIN LIBERO ALOHA cube transfer
Benchmarks: CALVIN ABC->D CALVIN ABCD->D LIBERO-Spatial LIBERO-Object LIBERO-Goal LIBERO-Long ALOHA cube transfer
Metrics: success rate average sequence length throughput (Hz) latency (s) MSE
Baselines: Diffusion Policy MDT OpenVLA 3DDA MoDE VPP Octo pi0 pi0-FAST ACT Binning-F FAST-AR
Builds on: Florence-2 ACT FAST OpenVLA-OFT Cox-de Boor recursion
Closest papers by meaning
- PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Control (ICML 2024 · similarity 0.86)
- NS-VLA: Towards Neuro-Symbolic Vision-Language-Action Models (NeurIPS 2026 · similarity 0.85)
- Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Sampling (ICLR 2025 · similarity 0.85)
- FineVLA: Fine-Grained Instruction Alignment for Steerable Vision-Language-Action Policies (NeurIPS 2026 · similarity 0.85)
- Quantization-Free Autoregressive Action Transformer (NeurIPS 2025 · similarity 0.85)
- QueST: Self-Supervised Skill Abstractions for Learning Continuous Control (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.