Research map / Large language models / AI and society

LLM-Powered Synthetic Environments for Self-Driving Scenarios

AAAI 2024 · Large language models › AI and society · approach: Critiques of LLM claims and judging · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

A short AAAI 2024 proposal outlining using GPT-4 to generate configuration parameters for game engines (Unity/Unreal) to create synthetic self-driving scenarios, with a planned evaluation framework but no implementation or results.

Problem

Generating realistic, diverse, and challenging synthetic environments, including rare edge cases, for testing and training autonomous vehicles is costly and limited; the paper proposes LLMs as a way to automate this.

Approach

Use text-to-text LLMs such as GPT-4 to convert natural-language scenario descriptions into game-engine configuration parameters (vehicle dynamics, environmental conditions), dynamically configuring Unity/Unreal scenes with user involvement and feedback, and proposing evaluation via realism, safety metrics, complexity, and performance improvement.

Key results

• No experiments or quantitative results are reported; the paper is a research proposal.

Limitations

• No implementation or experimental validation is presented.
• Potential bias in LLM-generated scenarios and LLM difficulty understanding complex scenarios are acknowledged but not addressed empirically.
• Evaluation plan is described only at a high level without concrete protocols.

Experimental setup

Metrics: collision rate emergency braking incidents complexity index decision-making speed user satisfaction

Builds on: SceneGen

Closest papers by meaning

• LaMPilot: An Open Benchmark Dataset for Autonomous Driving with Language Model Programs (CVPR 2024 · similarity 0.87)
• Driving Everywhere with Large Language Model Policy Adaptation (CVPR 2024 · similarity 0.86)
• Hybrid-Driving: An Autonomous Driving Decision Framework Integrating Large Language Models, Knowledge Graphs and Driving Rules (AAAI 2025 · similarity 0.86)
• ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehicles (CVPR 2024 · similarity 0.86)
• DriveGPT4-V2: Harnessing Large Language Model Capabilities for Enhanced Closed-Loop Autonomous Driving (CVPR 2025 · similarity 0.86)
• CoC-VLA: Delving into Adversarial Domain Transfer for Explainable Autonomous Driving via Chain-of-Causality Visual-Language-Action Model (NeurIPS 2025 · similarity 0.85)