Research map / Label-efficient and robust learning / Recognition and long-tail

DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving

ICLR 2025 · Label-efficient and robust learning › Recognition and long-tail · approach: Dense features and foundation priors · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DriveTransformer, a pure-attention end-to-end driving model that runs agent, map, and ego queries in parallel with sensor cross-attention, task self-attention, and streaming temporal cross-attention, trained in one stage.

Problem

Existing end-to-end autonomous driving methods chain perception, prediction, and planning sequentially, causing cumulative errors, multi-stage training, and costly dense BEV representations for long-range perception and temporal fusion.

Approach

Replace the sequential, BEV-based pipeline with a unified Transformer where all task queries (agent, map, ego) interact via task self-attention each block, attend directly to raw sensor tokens with 3D position encodings, and fuse history via FIFO queues of task queries, with a multi-mode planning head.

Key results

• DriveTransformer-Large reaches 63.46 Driving Score and 35.01% Success Rate on Bench2Drive, versus 45.81 and 16.36 for UniAD-Base.
• On nuScenes open-loop planning under the VAD metric, DriveTransformer-Large with ego status gets 0.33 average L2 and 0.07% average collision, versus 0.48 L2 for ParaDrive.
• Under camera crash, DriveTransformer loses only 2.9% driving score versus 9.2% for VAD-Base, suggesting better robustness; inference latency is 211.7ms versus 663.4ms for UniAD-Base.

Measured effects reported in the paper

Surprising findings

• Removing the temporal cross-attention causes only a small drop (60.45 to 56.22), so temporal information has the least influence in ablations.
• Pretraining the perception part does not improve results, supporting one-stage training over UniAD-style multi-stage training.

Limitations

• Tightly coupled sub-tasks make system maintenance and debugging harder.
• Still falls behind large vision-language pretrained backbones such as EVA02-CLIP-L in scaling study.
• Dev10 is a small 10-route proxy whose variance and representativeness are only partly checked.

Experimental setup

Datasets: Bench2Drive nuScenes

Benchmarks: Bench2Drive nuScenes Dev10

Metrics: Driving Score Success Rate Avg. L2 Collision rate Latency NDS mAP minADE mAP (online mapping) robustness driving score

Baselines: AD-MLP UniAD VAD ParaDrive TCP ThinkTwice DriveAdapter SparseDrive SparseAD BEVPlanner ST-P3

Builds on: StreamPETR DAB-DETR UniAD BEVFormer DiT

Closest papers by meaning

• Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Driving (NeurIPS 2024 · similarity 0.89)
• DualAD: Disentangling the Dynamic and Static World for End-to-End Driving (CVPR 2024 · similarity 0.88)
• VR-Drive: Viewpoint-Robust End-to-End Driving with Feed-Forward 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.88)
• Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Driving (ICLR 2025 · similarity 0.88)
• DriveGPT: Scaling Autoregressive Behavior Models for Driving (ICML 2025 · similarity 0.87)
• Prioritizing Perception-Guided Self-Supervision: A New Paradigm for Causal Modeling in End-to-End Autonomous Driving (NeurIPS 2025 · similarity 0.87)