Research map / Reinforcement learning / Visual and representation RL

Prompt to Transfer: Sim-to-Real Transfer for Traffic Signal Control with Prompt Learning

AAAI 2024 · Reinforcement learning › Visual and representation RL · approach: RL for combinatorial routing · 41 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PromptGAT, which uses GPT-4 prompted with weather, road type and traffic state to infer dynamics knowledge that augments a grounded action transformation forward model, reducing the sim-to-real gap for DQN traffic signal control in a CityFlow-to-SUMO setup.

Problem

Simulator-trained traffic signal control policies suffer a performance gap when deployed to real-world traffic because system dynamics differ between simulator and reality, and conventional grounded action transformation struggles with unobserved states such as extreme weather.

Approach

Use an LLM (GPT-4) via cloze-style prompts over weather, road type and traffic state to produce dynamics knowledge (acceleration, deceleration, emergency deceleration, startup delay), which is fused into the GAT forward model to ground simulator actions more accurately.

Key results

• In setting V4 (snowy), PromptGAT reduced ATT gap to 199.06 from 207.47 for Vanilla-GAT and 221.25 for Direct-Transfer.
• PromptGAT achieved the best ATT, reward, queue and delay in most settings V1-V4 relative to Vanilla-GAT, with ATT 154.97 vs 156.10 in V1.
• Improvement in forward-model accuracy was positively correlated with reduced real-world performance gap across metrics.

Measured effects reported in the paper

Surprising findings

• Vanilla-GAT and PromptGAT show nearly identical throughput gaps in some settings (e.g., V4 both 1750), so gains are not uniform across metrics.

Limitations

• Sim-to-sim setting (CityFlow to SUMO) rather than real-world deployment.
• Relies on a single proprietary LLM (GPT-4) with hand-designed prompts and small set of contexts.
• Gains over Vanilla-GAT are modest and within standard deviations in several metrics.

Experimental setup

Datasets: CityFlow simulator trajectories SUMO real-world configuration trajectories (simulated)

Benchmarks: LibSignal CityFlow-to-SUMO sim-to-real settings V1-V4

Metrics: average travel time throughput reward average queue length average delay

Baselines: Direct-Transfer Vanilla-GAT

Builds on: Grounded Action Transformation DQN chain-of-thought

Closest papers by meaning

• π-Light: Programmatic Interpretable Reinforcement Learning for Resource-Limited Traffic Signal Control (AAAI 2024 · similarity 0.87)
• RLFTSim: Realistic and Controllable Multi-Agent Traffic Simulation via Reinforcement Learning Fine-Tuning (CVPR 2026 · similarity 0.86)
• DiffLight: A Partial Rewards Conditioned Diffusion Model for Traffic Signal Control with Missing Data (NeurIPS 2024 · similarity 0.86)
• RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control (ICML 2025 · similarity 0.86)
• VLMLight: Safety-Critical Traffic Signal Control via Vision-Language Meta-Control and Dual-Branch Reasoning Architecture (NeurIPS 2025 · similarity 0.85)
• Advancing Multi-agent Traffic Simulation via R1-Style Reinforcement Fine-Tuning (ICLR 2026 · similarity 0.85)