Multi-Agent Deep Reinforcement Learning for Real-World Traffic Signal Controls - A Case Study

Increasing traffic congestion leads to significant costs, whereby poorly configured signaled intersections are a common bottleneck and root cause. Traditional traffic signal control (TSC) systems employ rule-based or heuristic methods to decide signal timings, while adaptive TSC solutions utilize a traffic-actuated control logic to increase their adaptability to real-time traffic changes. However, such systems are expensive to deploy and are often not flexible enough to adequately adapt to the volatility of today’s traffic dynamics. More recently, this problem became a frontier topic in the domain of deep reinforcement learning (DRL) and enabled the development of multi-agent DRL approaches that can operate in environments with several agents present, such as traffic systems with multiple signaled intersections. However, many of these proposed approaches were validated using artificial traffic grids. This paper presents a case study, where real-world traffic data from the town of Lemgo in Germany is used to create a realistic road model within VISSIM. A multi-agent DRL setup, comprising multiple independent deep Q-networks, is applied to the simulated traffic network. Traditional rule-based signal controls, modeled in LISA+ and currently employed in the real world at the studied intersections, are integrated into the traffic model and serve as a performance baseline. The performance evaluation indicates a significant reduction of traffic congestion when using the RL-based signal control policy over the conventional TSC approach with LISA+. Consequently, this paper reinforces the applicability of RL concepts in the domain of TSC engineering by employing a highly realistic traffic model.