Research map / Reinforcement learning / Visual and representation RL
Chain-of-Action: Trajectory Autoregressive Modeling for Robotic Manipulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Chain-of-Action (CoA), a visuomotor policy that generates robot action trajectories autoregressively in reverse from a predicted keyframe action back to the current state, with continuous action tokens, multi-token prediction, distance-based dynamic stop, and reverse temporal ensembling.
Problem
Forward next-step action prediction in visuomotor policies accumulates compounding errors and is myopic about long-horizon task success, so the paper seeks a trajectory-level modeling paradigm for manipulation.
Approach
Model the trajectory distribution in reverse: start from a keyframe (goal) action and autoregressively decode actions backward to the current gripper state, using continuous action tokens regularized by a latent consistency loss, MTP heads for local chunks, a distance-based stop, and a reverse-compatible temporal ensemble.
Key results
- CoA outperforms ACT by 16.3% average improvement on 81.7% of RLBench-60 tasks and Diffusion Policy by 23.2% on 80.0% of tasks.
- On RLBench-18 CoA averages 37.33 vs ACT 24.44 and DP 17.33, but remains well below 3D hierarchical methods (RVT-2 81.4).
- Real-world kitchen tasks: CoA achieves 0.613 average success rate vs ACT 0.463 and DP 0.363.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use reverse rather than forward action ordering (ablation) | CoA · RLBench-10 | success rate: 0.668 -> 0.756 | Helped |
| Keep keyframe anchoring but switch to forward action generation (ablation) | CoA · RLBench-10 | success rate: 0.756 -> 0.600 | Hurt |
| Replace latent consistency loss with action consistency loss (ablation) | CoA · RLBench-10 | success rate: 0.212 -> 0.756 | Helped |
| Remove reverse temporal ensembling (ablation) | CoA · RLBench-10 | success rate: 0.660 -> 0.756 | Helped |
| Use 1 MTP head instead of 5 (ablation) | CoA · RLBench-10 | success rate: 0.710 -> 0.756 | Helped |
| Use 2 MTP heads instead of 5 (ablation) | CoA · RLBench-10 | success rate: 0.704 -> 0.756 | Helped |
| Use 4 MTP heads instead of 5 (ablation) | CoA · RLBench-10 | success rate: 0.720 -> 0.756 | Helped |
| Use 8 MTP heads instead of 5 (ablation) | CoA · RLBench-10 | success rate: 0.672 -> 0.756 | Helped |
| Use 10 MTP heads instead of 5 (ablation) | CoA · RLBench-10 | success rate: 0.660 -> 0.756 | Helped |
| Append a keyframe action to the original ACT action chunk (ablation) | ACT · RLBench-10 | success rate: 0.488 -> 0.516 | Helped |
| Use CoA instead of ACT | RLBench-10 · CoA | success rate: +16.3% | Helped |
| Use CoA instead of Diffusion Policy | RLBench-60 · RLBench-60 | success rate: 23.2% | Helped |
Surprising findings
- ACT with an appended keyframe action (ACT+KF) gives only a marginal gain (0.516 vs 0.488), suggesting keyframe injection alone is insufficient.
- Replacing the latent consistency loss with direct action reconstruction drops success from 0.752 to 0.212.
Limitations
- Relies on keyframe heuristics (gripper state change or near-zero joint velocity) to split trajectories, which may not generalize to diverse task types.
- Still substantially below 3D-based hierarchical methods such as RVT-2 on RLBench-18.
- Real-world evaluation limited to 8 kitchen tasks with 10 trials each.
Experimental setup
Datasets: RLBench-60 RLBench-18 RLBench-10
Benchmarks: RLBench-60 RLBench-18 RLBench-10 real-world Fetch kitchen tasks
Metrics: success rate
Baselines: ACT Diffusion Policy Octo PerAct 3D Diffuser Actor RVT-2
Builds on: ACT C2F-ARM Diffusion Policy multi-token prediction
Closest papers by meaning
- Chain-of-Correction: Progress-Aware Policy Steering via Anchor-Grounded Predictive Reasoning (NeurIPS 2026 · similarity 0.90)
- Chain of World: World Model Thinking in Latent Motion (CVPR 2026 · similarity 0.90)
- FreqPolicy: Frequency Autoregressive Visuomotor Policy with Continuous Tokens (NeurIPS 2025 · similarity 0.89)
- Beyond World-Frame Action Heads: Motion-Centric Action Frames for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.89)
- PIVOT-R: Primitive-Driven Waypoint-Aware World Model for Robotic Manipulation (NeurIPS 2024 · similarity 0.89)
- FreqPolicy: Efficient Flow-based Visuomotor Policy via Frequency Consistency (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.