Adaptive Deep Q-Network Algorithm with Exponential Reward Mechanism for Traffic Control in Urban Intersection Networks

Muhammad Riza Tanwirul Fuad, Eric Okto Fernandez, Faqihza Mukhlish, Adiyana Putri, Herman Yoseph Sutarto, Yosi Agustina Hidayat and Endra Joelianto ()
Additional contact information
Muhammad Riza Tanwirul Fuad: Department of Engineering Physics, Faculty of Industrial Technology, Institut Teknologi Bandung, Bandung 40132, Indonesia
Eric Okto Fernandez: Department of Engineering Physics, Faculty of Industrial Technology, Institut Teknologi Bandung, Bandung 40132, Indonesia
Faqihza Mukhlish: Engineering Physics Research Group, Faculty of Industrial Technology, Institut Teknologi Bandung, Bandung 40132, Indonesia
Adiyana Putri: Graduate Program of Engineering Physics, Faculty of Industrial Technology, Institut Teknologi Bandung, Bandung 40132, Indonesia
Herman Yoseph Sutarto: Department of Intelligent System, PT. Pusat Riset Energi, Bandung 40226, Indonesia
Yosi Agustina Hidayat: Industrial System and Techno-Economy Research Group, Faculty of Industrial Technology, Institut Teknologi Bandung, Bandung 40132, Indonesia
Endra Joelianto: Instrumentation and Control Research Group, Faculty of Industrial Technology, Institut Teknologi Bandung, Bandung 40132, Indonesia

Sustainability, 2022, vol. 14, issue 21, 1-20

Abstract: The demand for transportation has increased significantly in recent decades in line with the increasing demand for passenger and freight mobility, especially in urban areas. One of the most negative impacts is the increasing level of traffic congestion. A possible short-term solution to solve this problem is to utilize a traffic control system. However, most traffic control systems still use classical control algorithms with the green phase sequence determined, based on a specific strategy. Studies have proven that this approach does not provide the expected congestion solution. In this paper, an adaptive traffic controller was developed that uses a reinforcement learning algorithm called deep Q-network (DQN). Since the DQN performance is determined by reward selection, an exponential reward function, based on the macroscopic fundamental diagram (MFD) of the distribution of vehicle density at intersections was considered. The action taken by the DQN is determining traffic phases, based on various rewards, ranging from pressure to adaptive loading of pressure and queue length. The reinforcement learning algorithm was then applied to the SUMO traffic simulation software to assess the effectiveness of the proposed strategy. The DQN-based control algorithm with the adaptive reward mechanism achieved the best performance with a vehicle throughput of 56,384 vehicles, followed by the classical and conventional control methods, such as Webster (50,366 vehicles), max-pressure (50,541 vehicles) and uniform (46,241 vehicles) traffic control. The significant increase in vehicle throughput achieved by the adaptive DQN-based control algorithm with an exponential reward mechanism means that the proposed traffic control could increase the area productivity, implying that the intersections could accommodate more vehicles so that the possibility of congestion was reduced. The algorithm performed remarkably in preventing congestion in a traffic network model of Central Jakarta as one of the world’s most congested cities. This result indicates that traffic control design using MFD as a performance measure can be a successful future direction in the development of reinforcement learning for traffic control systems.

Keywords: traffic control; reinforcement learning; deep Q-network; pressure; queue length; vehicle density; adaptive reward mechanism; macroscopic fundamental diagram (search for similar items in EconPapers)
JEL-codes: O13 Q Q0 Q2 Q3 Q5 Q56 (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2071-1050/14/21/14590/pdf (application/pdf)
https://www.mdpi.com/2071-1050/14/21/14590/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jsusta:v:14:y:2022:i:21:p:14590-:d:964839

Access Statistics for this article

Sustainability is currently edited by Ms. Alexandra Wu

More articles in Sustainability from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().