Research map / Reinforcement learning / Visual and representation RL
Quantization-Free Autoregressive Action Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Q-FAT, a quantization-free autoregressive transformer policy that predicts a continuous Gaussian mixture over actions (GIVT-style), plus mode-tracking sampling to reduce trajectory noise.
Problem
Autoregressive transformer imitation learning relies on discretizing or VQ-encoding continuous actions, which breaks the action space's continuous structure and complicates the pipeline; diffusion policies are slow and lack exact likelihoods.
Approach
Adapt GIVT to behavioral cloning: a decoder-only transformer directly outputs parameters of a diagonal-covariance Gaussian mixture over continuous actions, trained by negative log-likelihood with teacher forcing. Adds variance down-scaling and a mean-shift-based mode-finding sampler that samples modes weighted by Laplace-approximated mass.
Key results
- Q-FAT achieves 0.80 final IoU on PushT vs 0.78 for VQ-BeT and 0.74 for DiffPolicy-T, and 3.84/4 goals on Kitchen vs 3.66 for VQ-BeT.
- On conditional UR3 BlockPush, Q-FAT reaches 1.96/2 goals vs 1.94 for VQ-BeT; on conditional Kitchen it ties VQ-BeT at 3.78.
- Inference on a CPU is about 2x faster than VQ-BeT (~1.1 ms vs ~2.2 ms) due to a single linear action decoding layer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Q-FAT instead of VQ-BeT | Unconditional action policy · PushT | Final IoU: 0.78 -> 0.80 | Helped |
| Use Q-FAT instead of VQ-BeT | Unconditional action policy · Kitchen | Goals: 3.66 -> 3.84 | Helped |
| Use Q-FAT instead of VQ-BeT | Goal-conditioned policy · Cond. Kitchen | Goals: 3.78 -> 3.78 | No clear effect |
| Use Q-FAT instead of VQ-BeT | Trajectory prediction policy · nuScenes | L2 Error (m): 0.73 -> 0.75 | Hurt |
| Use Q-FAT instead of VQ-BeT | Trajectory prediction policy · nuScenes | Collision Rate (%): 0.29 -> 0.37 | Hurt |
| Down-scale Gaussian component variances compared with direct GMM sampling (ablation) | Q-FAT · Kitchen | Task success: 7% | Helped |
| Down-scale Gaussian component variances compared with direct GMM sampling (ablation) | Q-FAT · PushT | Task success: 6.5% | Helped |
| Down-scale Gaussian component variances compared with direct GMM sampling (ablation) | Q-FAT · Kitchen | Behavioral diversity: 2% | Hurt |
| Use mode sampling instead of variance down-scaling (ablation) | Q-FAT · Evaluated environments | Task success | No clear effect |
| Increase mixture count from 2 to 4 (ablation) | Q-FAT · Kitchen | Behavioral entropy: 18% | Helped |
| Vary mixture count k among 2, 4, 8, and 16 (ablation) | Q-FAT · Kitchen and UR3 BlockPush | Task success | No clear effect |
| Remove teacher forcing and use a full attention mask (ablation) | Q-FAT · Simulated robotics tasks | Policy performance | Hurt |
Surprising findings
- Down-scaling variances gives a measurable success gain, but mode sampling matched it on the benchmark metrics, the authors attributing this to well-separated mixture components.
- Roughly 70% of a rollout is unimodal according to the number of active modes, suggesting multimodality is needed only a minority of the time.
Limitations
- GMM loss assumes Euclidean action space, which may be problematic for Riemannian action spaces such as humanoids.
- Mode sampling showed no significant metric gains over variance down-scaling on the evaluated benchmarks.
- Diagonal covariance assumption; nuScenes results are only preliminary and VQ-BeT outperforms Q-FAT there.
Experimental setup
Datasets: PushT Franka Kitchen UR3 BlockPush BlockPush Multimodal Ant nuScenes
Benchmarks: PushT Image PushT Kitchen Image Kitchen Multimodal Ant UR3 BlockPush BlockPush Cond. Kitchen Cond. UR3 BlockPush nuScenes
Metrics: final IoU goals achieved per rollout behavioral entropy L2 error collision rate
Baselines: BeT VQ-BeT Diffusion Policy (CNN) Diffusion Policy (transformer) C-BeT BESO C-BESO CFG-BESO
Builds on: GIVT VQ-BeT minGPT Diffusion Policy
Closest papers by meaning
- QueST: Self-Supervised Skill Abstractions for Learning Continuous Control (NeurIPS 2024 · similarity 0.87)
- Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Training (NeurIPS 2024 · similarity 0.86)
- BAKU: An Efficient Transformer for Multi-Task Policy Learning (NeurIPS 2024 · similarity 0.86)
- Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model (ICLR 2025 · similarity 0.86)
- Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation (ICLR 2025 · similarity 0.86)
- Improving Generative Behavior Cloning via Self-Guidance and Adaptive Chunking (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.