Research map / Reinforcement learning / Imitation learning
Bidirectional Decoding: Improving Action Chunking via Guided Test-Time Sampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes the consistency-reactivity tradeoff of action chunking in imitation learning and proposes Bidirectional Decoding (BID), a test-time sample-selection method that combines backward coherence with forward contrast against strong and weak policy checkpoints.
Problem
Action chunking in generative behavior cloning has inconsistent effects: it captures temporal dependencies in demonstrations but reduces reactivity to unexpected states, and the choice of action horizon involves an unclear tradeoff.
Approach
At each step, sample multiple action chunks and select one by a backward criterion (closeness to the previously executed chunk) plus a forward contrast criterion (close to samples from a stronger checkpoint, far from samples from a weaker early checkpoint), giving test-time consistency and reactivity without changing the policy.
Key results
- BID gives about 32% relative improvement over the vanilla Diffusion Policy baseline across seven simulation tasks, and 46% when combined with EMA.
- On VQ-BeT Push-T with stochastic noise, closed-loop BID averages 43.8% success versus 38.9% for vanilla closed-loop sampling.
- In real-world dynamic picking, BID achieves at least 2x the success rate of the other compared methods in the dynamic setting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use BID instead of vanilla sampling | VQ-BeT · Push-T, closed-loop | success rate: 38.9 $\pm$ 1.5 -> 43.8 $\pm$ 1.4 | Helped |
| increase BID sample size from 1 to 32 | VQ-BeT · Push-T | success rate: 49.1 -> 54.4 | Helped |
| add forward contrast to backward coherence (ablation) | VQ-BeT BID · Push-T, open-loop average | success rate: 35.1 $\pm$ 1.0 -> 37.8 $\pm$ 1.6 | Helped |
| add negative samples to forward contrast (ablation) | VQ-BeT BID · Push-T, open-loop average | success rate: 37.0 $\pm$ 0.9 -> 37.8 $\pm$ 1.6 | Helped |
| add positive samples to forward contrast (ablation) | VQ-BeT BID · Push-T, open-loop average | success rate: 35.5 $\pm$ 1.2 -> 37.8 $\pm$ 1.6 | Helped |
| add forward contrast to backward coherence (ablation) | VQ-BeT BID · Push-T, closed-loop average | success rate: 41.4 $\pm$ 1.2 -> 43.8 $\pm$ 1.4 | Helped |
| add negative samples to forward contrast (ablation) | VQ-BeT BID · Push-T, closed-loop average | success rate: 42.8 $\pm$ 1.4 -> 43.8 $\pm$ 1.4 | Helped |
| add positive samples to forward contrast (ablation) | VQ-BeT BID · Push-T, closed-loop average | success rate: 42.7 $\pm$ 1.0 -> 43.8 $\pm$ 1.4 | Helped |
| use L1 instead of L2 distance in BID (ablation) | Diffusion Policy · Lift | success rate: 0.58 $\pm$ 0.06 -> 0.68 $\pm$ 0.05 | Helped |
| increase action horizon from 1 to 10 | one-dimensional imitation policy · One-dimensional simulation, noise 0.0 | total variation distance: 4.03 -> 0.00 | Helped |
| set action horizon to 1 rather than longer horizons | one-dimensional imitation policy · One-dimensional simulation, noise 0.4 | total variation distance: 0.36 vs 0.42, 0.59, 0.835, 1.11 | Helped |
| extend context length beyond the short context setting (ablation) | Diffusion Policy · Push-T | performance | Mixed |
Surprising findings
- Warmstart and EMA degrade performance on some tasks, showing that existing inference methods are not robust across tasks.
- Extending context length beyond short horizons hurts diffusion policy performance, while extending action horizon helps.
Limitations
- Computational overhead: roughly doubles inference time (13.2 ms to 25.2 ms at batch size 8), which may be costly for high-frequency low-cost robots.
- Analysis and method limited to policies with short context lengths.
- Forward contrast requires an additional weaker early checkpoint; the dynamic picking task omitted negative samples because no early checkpoint was available.
Experimental setup
Datasets: Push-T RoboMimic Franka Kitchen LeRobot Push-T checkpoint UMI diffusion policy checkpoint
Benchmarks: Push-T RoboMimic Lift RoboMimic Can RoboMimic Square RoboMimic Transport RoboMimic Tool Hang Franka Kitchen real-world dynamic placing real-world dynamic picking
Metrics: success rate total variation distance inference time
Baselines: Vanilla Warmstart EMA (temporal ensembling) VQ-BeT vanilla sampling
Builds on: Diffusion Policy receding horizon control temporal ensembling contrastive decoding
Closest papers by meaning
- PACE: Phase-Aware Chunk Execution for Robot Policies with Action Chunking (NeurIPS 2026 · similarity 0.89)
- Reinforcement Learning with Action Chunking (NeurIPS 2025 · similarity 0.89)
- Improving Generative Behavior Cloning via Self-Guidance and Adaptive Chunking (NeurIPS 2025 · similarity 0.88)
- FASTER: Rethinking Real-Time Flow VLAs (NeurIPS 2026 · similarity 0.88)
- SkiP: When to Skip and When to Refine for Efficient Robot Manipulation (NeurIPS 2026 · similarity 0.87)
- Real-Time Execution of Action Chunking Flow Policies (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.