Research map / Probabilistic and scientific modeling / Dynamical systems
Real-Time Execution of Action Chunking Flow Policies
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Real-Time Chunking, an inference-time inpainting method with soft masking and guidance clipping that enables smooth asynchronous execution of diffusion/flow action-chunking VLAs such as pi0.5.
Problem
High inference latency of large flow/diffusion-based VLAs causes pauses and jerky out-of-distribution motions at action chunk boundaries when executed synchronously or naively asynchronously.
Approach
Treat asynchronous chunking as inpainting: freeze actions guaranteed to execute during inference delay and softly inpaint the overlapping remainder of the new chunk via pseudoinverse-guidance with exponentially decaying soft mask weights and clipped guidance weights, requiring no retraining.
Key results
- In simulation, the method outperforms BID and temporal ensembling across inference delays, with the gap widening as delay increases; soft masking improves over hard masking at low delays.
- On real bimanual tasks with pi0.5, the method achieves the best average throughput at all inference delays, with statistically significant gains at +100ms and +200ms injected latency.
- The method shows no degradation under injected delay, whereas synchronous inference degrades linearly and temporal ensembling variants trigger protective stops.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use asynchronous inpainting with soft masking instead of hard masking (ablation) | Real-Time Chunking · Kinetix dynamic tasks | solve rate | Helped |
| Use exponential decay for soft-mask weights instead of other tested schedules (ablation) | Real-Time Chunking · Kinetix dynamic tasks | solve rate | Helped |
| Use guidance-based inpainting instead of Diffuser-style action overwriting (ablation) | Real-Time Chunking · Kinetix dynamic tasks | solve rate | Helped |
| Increase guidance clipping beyond β = 5 (ablation) | Real-Time Chunking · Kinetix simulated benchmark | performance | No clear effect |
| Use higher guidance clipping β (ablation) | Real-Time Chunking · PiZeroFive action chunks | maximum acceleration | Hurt |
| Add guidance-weight clipping (ablation) | Real-Time Chunking · Flow-policy action generation with few denoising steps | action chunk stability | Helped |
| Use Real-Time Chunking instead of baseline methods | PiZeroFive · Real-world tasks across inference delays | average task throughput | Helped |
| Use Real-Time Chunking under injected inference delay | PiZeroFive · Real-world tasks at +100ms and +200ms injected latency | average task throughput | Helped |
| Use Real-Time Chunking instead of synchronous inference | PiZeroFive · Real-world task execution with inference pauses removed | task completion progress | Helped |
| Use Real-Time Chunking instead of synchronous inference | PiZeroFive · Light candle | final task score | Helped |
| Use Real-Time Chunking instead of synchronous inference | PiZeroFive · Kinetix dynamic tasks as inference delay increases | solve rate | Helped |
| Use temporal ensembling instead of direct chunk execution | Action chunking policies · Kinetix dynamic tasks | solve rate | Hurt |
Surprising findings
- Temporal ensembling performs poorly even at zero inference delay in the multimodal simulated benchmark, since averages of valid actions need not be valid.
Limitations
- Adds significant computational overhead compared with sampling directly from the base policy.
- Applicable only to diffusion- and flow-based policies.
- Real-world experiments do not cover highly dynamic settings such as legged locomotion.
Experimental setup
Datasets: Kinetix simulated expert rollouts (1M transitions per environment) real-world bimanual robot demonstrations
Benchmarks: Kinetix dynamic benchmark (12 environments) 6 real-world bimanual manipulation tasks
Metrics: solve rate task progress average throughput max acceleration (jerkiness)
Baselines: Synchronous inference Naive async Bidirectional decoding (BID) Temporal ensembling (TE)
Builds on: pi0.5 PiGDM Diffuser temporal ensembling bidirectional decoding
Closest papers by meaning
- FASTER: Rethinking Real-Time Flow VLAs (NeurIPS 2026 · similarity 0.90)
- PACE: Phase-Aware Chunk Execution for Robot Policies with Action Chunking (NeurIPS 2026 · similarity 0.90)
- Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Sampling (ICLR 2025 · similarity 0.87)
- Reinforcement Learning with Action Chunking (NeurIPS 2025 · similarity 0.87)
- LongScape: Advancing Long-Horizon Embodied World Models with Context-Aware MoE (NeurIPS 2026 · similarity 0.86)
- ElegantVLA: Learning When to Think for Efficient Vision-Language-Action Models (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.