Research map / Large language models / Agents, code and math
Driving Everywhere with Large Language Model Policy Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
LLaDA is a training-free framework that uses GPT-4 with a keyword-based traffic rule extractor over local driver handbooks to adapt human or autonomous driving plans to local traffic rules.
Problem
Autonomous vehicles and human drivers struggle to adapt driving behavior to new locations' traffic rules and customs, which limits AV deployment beyond geo-fenced areas.
Approach
Feed a nominal plan, scene description, unexpected situation, and the local driver handbook into a frozen LLM after a Traffic Rule Extractor filters handbook paragraphs by LLM-extracted keywords, yielding an adapted natural-language plan usable by human drivers or AV planners such as GPT-Driver.
Key results
- On nuScenes Boston, GPT-Driver + LLaDA re-planning reduces average L2 error from 0.63 m to 0.62 m and average collision rate from 0.58% to 0.56%.
- In a user study with 24 participants over 8 cases, 70.3% said instructions strictly follow local law and 82.8% found them very or extremely helpful.
- In safety tests over 50 examples per case, LLaDA showed a 0% error rate on the four critical-case categories.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LLaDA re-planning to GPT-Driver | GPT-Driver · nuScenes Boston validation, Avg. | L2 (m): 0.63 -> 0.62 | Helped |
| Add LLaDA re-planning to GPT-Driver | GPT-Driver · nuScenes Boston validation, Avg. | Collision (%): 0.58 -> 0.56 | Helped |
| Add LLaDA re-planning to GPT-Driver | GPT-Driver · nuScenes Boston validation, 1s | L2 (m): 0.27 -> 0.27 | No clear effect |
| Add LLaDA re-planning to GPT-Driver | GPT-Driver · nuScenes Boston validation, 2s | L2 (m): 0.59 -> 0.58 | Helped |
| Add LLaDA re-planning to GPT-Driver | GPT-Driver · nuScenes Boston validation, 3s | L2 (m): 1.04 -> 1.02 | Helped |
| Add LLaDA re-planning to GPT-Driver | GPT-Driver · nuScenes Boston validation, 1s | Collision (%): 0.22 -> 0.22 | No clear effect |
| Add LLaDA re-planning to GPT-Driver | GPT-Driver · nuScenes Boston validation, 2s | Collision (%): 0.45 -> 0.41 | Helped |
| Add LLaDA re-planning to GPT-Driver | GPT-Driver · nuScenes Boston validation, 3s | Collision (%): 1.07 -> 1.04 | Helped |
Surprising findings
- The motion planning improvement on nuScenes is marginal (L2 0.63 to 0.62 m; collision 0.58% to 0.56%).
Limitations
- LLM runtime is too high for closed-loop AV control
- Performance is sensitive to the quality of scene descriptions, which GPT-4V sometimes gets wrong
- Requires a human or VLM to detect unexpected situations; no automatic detector
Experimental setup
Datasets: nuScenes nuPlan driver handbooks (California, NYC, Singapore, Boston, Germany, etc.)
Benchmarks: nuScenes Boston validation subset
Metrics: L2 error (m) collision rate (%) user study: instruction follows local law and usefulness
Baselines: GPT-Driver
Builds on: GPT-Driver Code-As-Policies ITP
Closest papers by meaning
- Hybrid-Driving: An Autonomous Driving Decision Framework Integrating Large Language Models, Knowledge Graphs and Driving Rules (AAAI 2025 · similarity 0.88)
- AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning (NeurIPS 2025 · similarity 0.87)
- LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs (CVPR 2024 · similarity 0.87)
- LLM-Powered Synthetic Environments for Self-Driving Scenarios (AAAI 2024 · similarity 0.86)
- Distilling Multi-modal Large Language Models for Autonomous Driving (CVPR 2025 · similarity 0.86)
- Less Language, More Latents: Annotation-Efficient VLAs for Driving (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.