Research map / Vision-language and multimodal / Robot action and navigation

RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation

ICLR 2025 · Vision-language and multimodal › Robot action and navigation · approach: Memory and subgoal-based manipulation · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces RDT-1B, a 1.2B-parameter diffusion transformer foundation model for language-conditioned bimanual manipulation, pre-trained on 46 multi-robot datasets with a physically interpretable 128-dim unified action space and fine-tuned on a 6K-episode ALOHA dataset.

Problem

Building generalizable bimanual manipulation foundation models is hard due to scarce bimanual data and multi-modal action distributions from coordinating two arms, plus heterogeneity across robots' action spaces.

Approach

Combines a diffusion-based scalable Transformer (DiT with QKNorm, RMSNorm, MLP decoder, alternating cross-attention conditioning) with a physically interpretable unified action space that maps heterogeneous robot actions into one 128-dim layout for cross-robot pre-training.

Key results

• RDT (ours) reaches 100% total success on Pour Water-L-1/3 instruction following and 75% correct-hand/amount-level subtask results, while ACT, OpenVLA, and Octo mostly score 0 on unseen-object and unseen-scene tasks.
• On Fold Shorts (1-shot), RDT achieves 68% success versus 40% for RDT (scratch) and 0-4% for baselines.
• Robot Dog dexterity task: RDT reaches 48% success on the hardest sub-metric, versus 32% for RDT (scratch) and 0% for OpenVLA and Octo; the paper reports a 56% average improvement over baselines.

Measured effects reported in the paper

Surprising findings

• Regression-based RDT variant without diffusion performs worse on instruction following (12.5%) than diffusion-based variants, and a smaller 166M model drops on unseen object generalization.
• OpenVLA and Octo achieve near-zero success on most bimanual tasks despite large pre-training, while the simple ACT baseline is competitive on some seen-object tasks.

Limitations

• Evaluation limited to 7 real-robot tasks on a single ALOHA platform with few trials per task (8 or 25).
• Fine-tuning is needed for embodiment gap; zero-shot transfer to the target robot without fine-tuning is not established.
• Pre-training mostly single-arm data; benefits for bimanual-specific pretraining are not isolated.

Experimental setup

Datasets: Open X-Embodiment RT-1 Dataset DROID RH20T Mobile ALOHA Dataset RoboSet BridgeData V2 Self-collected Mobile ALOHA bimanual fine-tuning dataset

Benchmarks: Wash Cup Pour Water Pour Water-L-1/3 Pour Water-R-2/3 Handover Fold Shorts Robot Dog

Metrics: success rate

Baselines: ACT OpenVLA Octo RDT (regress) RDT (small) RDT (scratch)

Builds on: diffusion transformer DPM-Solver++ Action Chunking with Transformers Open X-Embodiment

Closest papers by meaning

• Spatial-Temporal Graph Diffusion Policy with Kinematic Modeling for Bimanual Robotic Manipulation (CVPR 2025 · similarity 0.90)
• Generate Subgoal Images before Act: Unlocking the Chain-of-Thought Reasoning in Diffusion Model for Robot Manipulation with Multimodal Prompts (CVPR 2024 · similarity 0.89)
• Robots Pre-train Robots: Manipulation-Centric Robotic Representation from Large-Scale Robot Datasets (ICLR 2025 · similarity 0.89)
• GravMAD: Grounded Spatial Value Maps Guided Action Diffusion for Generalized 3D Manipulation (ICLR 2025 · similarity 0.88)
• Diffusion-EDFs: Bi-equivariant Denoising Generative Modeling on SE(3) for Visual Robotic Manipulation (CVPR 2024 · similarity 0.88)
• ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learning (CVPR 2025 · similarity 0.88)