Research map / Label-efficient and robust learning / Recognition and long-tail
Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Driving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Bench2Drive, a closed-loop CARLA benchmark for end-to-end autonomous driving with a 2M-frame official expert training set (Think2Drive-collected), 220 short single-scenario routes across 44 scenarios, and multi-ability metrics, evaluated on several E2E-AD baselines.
Problem
Existing E2E-AD evaluation is either open-loop (nuScenes L2/collision, which poorly reflects planning) or closed-loop on long CARLA routes whose driving score has high variance, and methods use self-collected training data, preventing fair algorithm-level comparison.
Approach
Provide a closed-loop benchmark with short single-scenario routes (about 150 m) covering 44 interactive scenarios, paired with a standardized large expert dataset collected by the RL expert Think2Drive, and per-skill metrics (success rate, driving score, efficiency, comfortness) to disentangle driving abilities.
Key results
- DriveAdapter (with expert feature distillation) achieves the highest driving score of 64.22 and success rate 33.08% among baselines.
- AD-MLP obtains 0% success rate and 18.05 driving score, while UniAD-Base has the lowest open-loop L2 (0.73) yet a lower driving score (45.81) than the distilled methods.
- Methods with expert feature distillation (TCP*, ThinkTwice*, DriveAdapter*) outperform those without (VAD, UniAD) by a large margin; mean multi-ability score reaches 42.08 for DriveAdapter*.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add expert feature distillation to TCP-traj (ablation) | TCP-traj · Bench2Drive base | Driving Score: 49.30 -> 59.90 | Helped |
| Add expert feature distillation to TCP-traj (ablation) | TCP-traj · Bench2Drive base | Success Rate (%): 20.45 -> 30.00 | Helped |
| Add expert feature distillation to TCP-traj (ablation) | TCP-traj · Bench2Drive base | Efficiency: 78.78 -> 76.54 | Hurt |
| Use trajectory output rather than control output (ablation) | TCP* · Bench2Drive base | Driving Score: 30.47 -> 59.90 | Helped |
| Rely on open-loop L2 to rank methods with good dataset fit | E2E-AD methods · Bench2Drive base | Driving Score: 45.81 vs 42.35 | Mixed |
Surprising findings
- Open-loop L2 error does not reflect closed-loop performance: UniAD-Base has lower L2 than VAD but worse driving score.
- AD-MLP fails to reach a decent L2 on Bench2Drive, unlike findings on nuScenes, attributed to greater behavior diversity.
Limitations
- CARLA rendering has a gap with real-world imagery, limiting realism of sensor data.
- Interactive scenarios are underrepresented in the training data, so all models perform poorly on merging, overtaking, and emergency braking.
- Evaluation relies on simulation, so simulation-to-reality gap remains.
Experimental setup
Datasets: Bench2Drive official training set (CARLA v2 expert data)
Benchmarks: Bench2Drive 220-route closed-loop evaluation
Metrics: Driving Score Success Rate Efficiency Comfortness Avg. L2 per-skill scores (Merging, Overtaking, Emergency Brake, Give
Baselines: AD-MLP UniAD VAD TCP ThinkTwice DriveAdapter BEVFormer
Builds on: Think2Drive CARLA Leaderboard v2 nuPlan smoothness protocol nuScenes sensor configuration
Closest papers by meaning
- NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking (NeurIPS 2024 · similarity 0.89)
- DriveHierarchy: A Benchmark for Diagnosing VLM Driving Capabilities from Open-Loop Understanding to Closed-Loop Execution (NeurIPS 2026 · similarity 0.89)
- DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving (ICLR 2025 · similarity 0.89)
- Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving (NeurIPS 2026 · similarity 0.88)
- Autonomous Driving Research Requires a Community-Driven Data Paradigm (NeurIPS 2026 · similarity 0.88)
- Model-Based Policy Adaptation for Closed-Loop End-to-end Autonomous Driving (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.