Research map / Reinforcement learning / Visual and representation RL
Model-Based Policy Adaptation for Closed-Loop End-to-end Autonomous Driving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Model-based Policy Adaptation (MPA), which generates counterfactual driving data in a 3DGS simulator, trains a diffusion residual policy adapter and a multi-principle Q-value model, and uses Q-guided selection among sampled trajectories to improve closed-loop E2E driving on nuScenes via HUGSIM.
Problem
End-to-end driving policies trained by open-loop imitation suffer compounding errors and poor generalization in closed-loop deployment, due to observation mismatch and objective mismatch.
Approach
Generate counterfactual trajectories with a 3DGS-based simulator, train a diffusion residual adapter on top of a frozen base E2E policy plus a multi-step Q-value model over interpretable driving principles, and at inference sample multiple residual proposals and pick the one with highest Q.
Key results
- In-domain, MPA (UniAD) reaches 93.6-94.9 RC and 66.4-67.0 HDScore versus 19.4 for UniAD and 46.7 for LTF.
- On safety-critical scenes, MPA (UniAD-based variant) reaches HDScore 70.4 with RC 95.1, versus 4.5 for UniAD and 24.2 for LTF.
- Ablations show removing the Q-route component drops HDScore to 5.1 on unseen nominal scenes, and longer counterfactual rollout steps improve test performance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply MPA to pretrained E2E agents | UniAD, VAD, and LTF · Safety-critical scenes | HDScore: 24.2 -> 70.4; 16.0 -> 74.7; 24.2 -> 56.3 | Helped |
| add the policy adapter (ablation) | MPA (UniAD) · Safety-critical scenes | HDScore: 55.3 -> 70.4 | Helped |
| add Q_route (ablation) | MPA (UniAD) · Unseen nominal scenes | HDScore: 5.1 -> 60.9 | Helped |
| add Q_dist (ablation) | MPA (UniAD) · Unseen nominal scenes | HDScore: 43.2 -> 60.9 | Helped |
| add Q_collision (ablation) | MPA (UniAD) · Safety-critical scenes | HDScore: 39.2 -> 70.4 | Helped |
| add Q_speed (ablation) | MPA (UniAD) · Safety-critical scenes | HDScore: 50.1 -> 70.4 | Helped |
| increase counterfactual rollout steps (ablation) | MPA · Unseen nominal scenes | test-time performance | Helped |
| increase policy-adapter mode capacity (ablation) | MPA · Safety-critical scenes | HDScore | Helped |
Surprising findings
- The paper reports that the observation mismatch is minor in open-loop evaluation, with 3DGS-rendered open-loop L2 errors close to ground truth.
- The adapter removal leads to similar drops for VAD and LTF, suggesting the value model plays a more critical role than the adapter in those cases.
Limitations
- Assumes reliable 3DGS rendering only under constrained trajectory deviations (lateral deviation under 0.6 m).
- Value modeling is decoupled from policy optimization; no online RL over the simulator.
- Evaluated only on nuScenes-based HUGSIM scenes with limited splits (70 in-domain, 70 unseen, 10 safety-critical scenes).
Experimental setup
Datasets: nuScenes HUGSIM counterfactual rollouts
Benchmarks: HUGSIM
Metrics: RC NC DAC TTC COM HDScore
Baselines: UniAD VAD LTF AD-MLP BC-Safe Diffusion
Builds on: UniAD VAD DDIM RAD HUGSIM
Closest papers by meaning
- Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Driving (NeurIPS 2024 · similarity 0.88)
- AdaWM: Adaptive World Model based Planning for Autonomous Driving (ICLR 2025 · similarity 0.87)
- MAPLE: Latent Multi-Agent Play for End-to-End Autonomous Driving (NeurIPS 2026 · similarity 0.87)
- AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.87)
- RAD: Training an End-to-End Driving Policy via Large-Scale 3DGS-based Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Prioritizing Perception-Guided Self-Supervision: A New Paradigm for Causal Modeling in End-to-End Autonomous Driving (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.