Research map / Reinforcement learning / Visual and representation RL

π-Light: Programmatic Interpretable Reinforcement Learning for Resource-Limited Traffic Signal Control

AAAI 2024 · Reinforcement learning › Visual and representation RL · approach: RL for combinatorial routing · 18 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes π-Light, which represents traffic signal control policies as programs written in a custom DSL and searches for them with MCTS (plus Bayesian optimization for constants), yielding interpretable, cross-region generalizable, edge-deployable controllers.

Problem

Deep RL traffic signal control policies are black-box, over-parameterized neural networks that lack interpretability, cannot run on resource-limited edge hardware, and generalize poorly across intersections and cities.

Approach

Represent the TSC policy as a non-differentiable program built from a DSL with if-then/if-then-else control flow, a MaxPressure-style base program, and discover programs via MCTS with UCT over program transformation rules, using the reciprocal of average travel time as a global reward; parameters are tuned with Bayesian optimization.

Key results

• On Hangzhou2 π-Light reaches 90.50 s travel time vs 102.87 s for TinyLight (second best), and on Los Angeles 437.42 s vs 489.93 s for TinyLight.
• In generalization tests, π-Light keeps stable performance, e.g. Hangzhou2→Manhattan 209.07 s travel time vs 315.83 s for MaxPressure and 1781.96 s for MPLight.
• π-Light's resource needs are slightly higher than MaxPressure's, and it is the only learning-based method on the performance-vs-resource Pareto frontier.

Measured effects reported in the paper

Surprising findings

• Programmatic policies with symbolic structure outperform expressive neural DRL policies in most environments.
• Standard DRL methods (FRAP, MPLight, TinyLight) degrade sharply or fall below MaxPressure when transferred across cities.

Limitations

• Program resource use is slightly higher than MaxPressure's.
• Program structure is constrained (max sequence length 6, depth 2) and interactions between features are omitted by design.

Experimental setup

Datasets: Hangzhou Manhattan Atlanta Jinan Los Angeles

Benchmarks: CityFlow Hangzhou1 CityFlow Hangzhou2 CityFlow Manhattan CityFlow Atlanta CityFlow Jinan CityFlow Los Angeles

Metrics: average travel time throughput FLOPs memory consumption

Baselines: FixedTime MaxPressure SOTL CoLight FRAP MPLight EcoLight TinyLight

Builds on: MaxPressure Monte Carlo tree search UCT Bayesian optimization DRSQ

Closest papers by meaning

• Toward A Thousand Lights: Decentralized Deep Reinforcement Learning for Large-Scale Traffic Signal Control (AAAI 2020 · similarity 0.91)
• VLMLight: Safety-Critical Traffic Signal Control via Vision-Language Meta-Control and Dual-Branch Reasoning Architecture (NeurIPS 2025 · similarity 0.88)
• RobustLight: Improving Robustness via Diffusion Reinforcement Learning for Traffic Signal Control (ICML 2025 · similarity 0.87)
• Prompt to Transfer: Sim-to-Real Transfer for Traffic Signal Control with Prompt Learning (AAAI 2024 · similarity 0.87)
• DiffLight: A Partial Rewards Conditioned Diffusion Model for Traffic Signal Control with Missing Data (NeurIPS 2024 · similarity 0.87)
• CoLLMLight: Cooperative Large Language Model Agents for Network-Wide Traffic Signal Control (ICLR 2026 · similarity 0.86)