Research map / Reinforcement learning / Visual and representation RL
RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RAD, a 3DGS-based closed-loop RL framework that post-trains an end-to-end driving policy with PPO plus IL regularization and auxiliary directional losses, reducing collision rate about 3x versus IL baselines on a 3DGS benchmark.
Problem
End-to-end autonomous driving policies trained by imitation learning suffer from causal confusion, the open-loop gap, and poor handling of safety-critical and out-of-distribution events.
Approach
Closed-loop RL in photorealistic 3DGS reconstructions of real driving scenes, with safety and deviation rewards, decoupled lateral/longitudinal discrete actions and separate PPO clipping, plus directional auxiliary losses and IL interleaved as a regularizer to keep human-like behavior.
Key results
- RAD achieves CR 0.089 versus 0.229 to 0.341 for IL-based baselines (TransFuser 0.320, VADv2 0.270), about 3x lower collision rate.
- Pure IL has CR 0.229 and ADD 0.238; pure RL has CR 0.143 and ADD 0.345; RL+IL gives CR 0.089 and ADD 0.257.
- Removing all auxiliary objectives raises CR to 0.187 versus 0.089 for the full setup.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RL post-training to an IL policy (ablation) | RAD · 3DGS evaluation benchmark | CR: 0.229 -> 0.089 | Helped |
| add IL regularization to an RL policy (ablation) | RAD · 3DGS evaluation benchmark | ADD: 0.345 -> 0.257 | Helped |
| remove dynamic collision reward (ablation) | RAD reward function · 3DGS evaluation benchmark | CR: 0.089 -> 0.238 | Hurt |
| use all four reward components (ablation) | RAD reward function · 3DGS evaluation benchmark | CR: 0.238 -> 0.089 | Helped |
| remove all auxiliary objectives (ablation) | RAD · 3DGS evaluation benchmark | CR: 0.089 -> 0.249 | Hurt |
| remove the PPO objective while retaining auxiliary objectives (ablation) | RAD · 3DGS evaluation benchmark | CR: 0.089 -> 0.187 | Hurt |
| replace IL with pure RL post-training (ablation) | RAD · 3DGS evaluation benchmark | CR: 0.229 -> 0.143 | Helped |
| replace IL+RL with pure RL post-training (ablation) | RAD · 3DGS evaluation benchmark | ADD: 0.257 -> 0.345 | Hurt |
| use RAD instead of VADv2 | end-to-end driving policy · 3DGS evaluation benchmark | CR: 0.270 -> 0.089 | Helped |
| use RAD instead of TransFuser | end-to-end driving policy · 3DGS evaluation benchmark | DR: 0.235 -> 0.063 | Helped |
Surprising findings
- Pure RL yields higher deviation from expert behavior (ADD 0.345) than pure IL, while RL+IL recovers near-IL fidelity with lower collisions.
- RAD has higher longitudinal jerk (4.495) than the IL baseline TransFuser (4.538 is higher; RAD is lower) but RAD's longitudinal jerk exceeds pure IL's 3.928 in the ablation table.
Limitations
- 3DGS rendering is weaker for non-rigid pedestrians, unobserved views, and low-light scenes.
- Evaluation is limited to a 3DGS benchmark of 337 clips built from the same data source as training.
Experimental setup
Datasets: 2000 hours of real-world expert driving demonstrations 3968 3DGS RL training clips 337 3DGS evaluation clips
Benchmarks: 337-clip 3DGS closed-loop evaluation benchmark
Metrics: CR DCR SCR DR PDR HDR ADD longitudinal jerk lateral jerk
Baselines: TransFuser VAD GenAD VADv2 IL-only training pure RL
Builds on: PPO Generalized Advantage Estimation StreetGaussian 3D Gaussian Splatting VAD
Closest papers by meaning
- Beyond Self-Play and Scale: A Behavior Benchmark for Generalization in Autonomous Driving (NeurIPS 2026 · similarity 0.88)
- Prioritizing Perception-Guided Self-Supervision: A New Paradigm for Causal Modeling in End-to-End Autonomous Driving (NeurIPS 2025 · similarity 0.88)
- Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2) (NeurIPS 2025 · similarity 0.87)
- DriveDPO: Policy Learning via Safety DPO For End-to-End Autonomous Driving (NeurIPS 2025 · similarity 0.87)
- Model-Based Policy Adaptation for Closed-Loop End-to-end Autonomous Driving (NeurIPS 2025 · similarity 0.87)
- RLfOLD: Reinforcement Learning from Online Demonstrations in Urban Autonomous Driving (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.