Research map / Reinforcement learning / Visual and representation RL
VLMLight: Safety-Critical Traffic Signal Control via Vision-Language Meta-Control and Dual-Branch Reasoning Architecture
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VLMLight, a traffic signal control framework that pairs a VLM scene describer and an LLM meta-controller that routes between a PPO-trained RL policy for routine traffic and a multi-agent LLM reasoning branch for emergency-vehicle cases, evaluated in a custom image-based SUMO simulator.
Problem
Traffic signal control must balance efficiency and safety, but RL controllers use vectorized states and static rewards that miss semantically rich, safety-critical cues such as emergency vehicles, while LLM-based reasoning is too slow for standalone control and existing simulators do not render imag
Approach
A safety-prioritized LLM meta-controller routes each intersection state, described by a VLM from multi-view images, either to a fast RL policy for routine traffic or to a deliberative branch of collaborating LLM agents (phase reasoning, signal planning, rule verification) for safety-critical cases such as emergency vehicles.
Key results
- VLMLight reduces AEWT on Songdo from 22.04 s (best RL, UniTSA) to 7.48 s, a reduction of about 66%.
- On routine traffic, VLMLight's ATT on Songdo is 87.14 s versus 86.80 s for UniTSA, an increase under 1%.
- Full inference latency with all agents is 11.48 s on Songdo, compared with a 13 s signal phase buffer.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VLMLight’s dual-branch meta-control and deliberative reasoning instead of Vanilla-VLM | Traffic signal control · Songdo | AEWT: 11.05 \pm 1.73 -> 7.48 \pm 0.45 | Helped |
| Use VLMLight’s dual-branch meta-control and deliberative reasoning instead of Vanilla-VLM | Traffic signal control · Yau Ma Tei | AETT: 27.79 \pm 3.79 -> 13.50 \pm 0.56 | Helped |
| Use VLMLight’s dual-branch meta-control and deliberative reasoning instead of Vanilla-VLM | Traffic signal control · Massy | AEWT: 3.04 \pm 0.61 -> 2.60 \pm 0.10 | Helped |
| Use VLMLight rather than the strongest RL baseline | Traffic signal control · Songdo | ATT: 86.80 \pm 4.89 -> 87.14 \pm 4.98 | Hurt |
| Add Agent Phase to the deliberative reasoning branch (ablation) | VLMLight · Songdo | AEWT: 9.03 -> 8.12 | Helped |
| Add Agent Phase to the deliberative reasoning branch (ablation) | VLMLight · Songdo | AETT: 58.93 -> 49.96 | Helped |
| Add Agent Check to the deliberative reasoning branch (ablation) | VLMLight · Songdo | AEWT: 8.12 -> 7.48 | Helped |
| Add Agent Check to the deliberative reasoning branch (ablation) | VLMLight · Songdo | AETT: 49.96 -> 48.88 | Helped |
| Add Agent Phase and Agent Check to the deliberative reasoning branch (ablation) | VLMLight · Songdo | Time (s): 7.10 -> 11.48 | Hurt |
Surprising findings
- Vanilla-VLM without meta-control performs worse than RL baselines on routine ATT (105.48 s on Songdo), indicating a single monolithic VLM-LLM pipeline is insufficient.
Limitations
- Simulator lacks diverse weather and lighting conditions, limiting visual robustness evaluation.
- All experiments are on single intersections; multi-intersection coordination is untested.
Experimental setup
Datasets: Songdo traffic footage Yau Ma Tei traffic footage Massy traffic footage
Benchmarks: Songdo (SUMO-based simulator) Yau Ma Tei (SUMO-based simulator) Massy (SUMO-based simulator)
Metrics: ATT AWT AETT AEWT
Baselines: FixTime Webster MaxPressure IntelliLight UniTSA A-CATs 3DQN-TSCC CCDA Vanilla-VLM
Builds on: PPO SUMO IntelliLight UniTSA
Closest papers by meaning
- π-Light: Programmatic Interpretable Reinforcement Learning for Resource-Limited Traffic Signal Control (AAAI 2024 · similarity 0.88)
- CoLLMLight: Cooperative Large Language Model Agents for Network-Wide Traffic Signal Control (ICLR 2026 · similarity 0.88)
- DriveCombo: Benchmarking Compositional Traffic Rule Reasoning in Autonomous Driving (CVPR 2026 · similarity 0.87)
- DriveAgent-R1: Advancing VLM-based Autonomous Driving with Active Perception and Hybrid Thinking (ICLR 2026 · similarity 0.87)
- Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning (AAAI 2026 · similarity 0.87)
- RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.