Comparative analysis of double deep Q-network (Double DQN) and Proximal Policy Optimization (PPO) for lane-keeping in autonomous driving

SABBIR M *

Department Of Artificial Intelligence, School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, China.
 
Research Article
International Journal of Science and Research Archive, 2024, 13(02), 1207–1222.
Article DOI: 10.30574/ijsra.2024.13.2.2237
Publication history: 
Received on 08 October 2024; revised on 16 November 2024; accepted on 19 November 2024
 
Abstract: 
Lane-keeping is a vital function in autonomous driving, important for vehicle safety, stability, and adherence to traffic flow. The intricacy of lane-keeping control resides in balancing precision and responsiveness across varied driving circumstances. This article gives a comparative examination of two reinforcement learning (RL) algorithms—Double Deep Q-Network (Double DQN) and Proximal Policy Optimization (PPO)—for lane-keeping across discrete and continuous action spaces. Double DQN, an upgrade of standard Deep Q-Networks, eliminates overestimation bias in Q-values, demonstrating its usefulness in discrete action spaces. This method shines in low-dimensional environments like highways, where lane-keeping requires frequent, discrete modifications. In contrast, PPO, a strong policy-gradient method built for continuous control, performs well in high-dimensional situations, such as urban roadways and curved highways, where continual, accurate steering changes are necessary. The methods were tested in MATLAB/Simulink simulations that simulate both highway and urban driving circumstances. Each model integrates vehicle dynamics and neural network topologies to build control techniques. Results demonstrate that Double DQN consistently maintains lane position in highway settings, exploiting its ability to minimize overestimations in Q-values, thereby attaining stable lane centering. PPO outshines in dynamic and unpredictable settings, managing continual control adjustments well, especially under difficult traffic conditions and on curving roadways. This study underscores the importance of matching RL algorithms to the action-space requirements of specific driving environments, with Double DQN excelling in discrete tasks and PPO in continuous adaptive control, contributing valuable insights toward enhancing the flexibility and safety of autonomous vehicles.
 
Keywords: 
Autonomous Driving; Lane-Keeping; Reinforcement Learning; Double Deep Q-Network (Double DQN); Proximal Policy Optimization (PPO); Action Space
 
Full text article in PDF: 
 
Copyright information: 

Copyright © 2024 Author(s) retain the copyright of this article. This article is published under the terms of the Creative Commons Attribution Liscense 4.0