JianQiang Yan , Yinxiang Li , Yuan Gao , BoTing Qu , Jing Chen
{"title":"Q_EDQ: Efficient path planning in multimodal travel scenarios based on reinforcement learning","authors":"JianQiang Yan , Yinxiang Li , Yuan Gao , BoTing Qu , Jing Chen","doi":"10.1016/j.tbs.2024.100943","DOIUrl":null,"url":null,"abstract":"<div><div>Recently, Mobility as a Service (MaaS) has garnered increasing attention by integrating various modes of transportation to provide users with a unified travel solution. However, In multimodal transportation planning, we primarily face three challenges: Firstly, a multimodal travel network is constructed that covers multiple travel modes and is highly scalable. Secondly, the routing algorithm fully considers the dynamic and real-time nature of the multimodal travel process. Finally, a generalized travel cost objective function is constructed that considers the psychological burden of transfers on passengers in multimodal travel scenarios. In this study, we firstly constructed an integrated multimodal transport network based on graph theory, which covers four transport modes, namely, the metro, the bus, the car-sharing and the walking. Subsequently, by introducing a double-Q learning mechanism and an optimized dynamic exploration strategy, we propose a new algorithm, Q_EDQ, the algorithm aims to learn the globally optimal path as efficiently as possible, with faster convergence speed and improved stability. Experiments utilizing real bus and metro data from Xi’an, Shaanxi Province, were conducted to compare the Q_EDQ algorithm with traditional genetic algorithms. In the conducted four experiments, compared to the optimal paths planned by traditional genetic algorithms, the improved Q-algorithm achieved a minimum efficiency increase of 12.52% and a maximum of 35%. These results demonstrate the enhanced capability of the improved Q-algorithm to learn globally optimal paths in complex multimodal transportation networks. Compared to the classical Q algorithm, the algorithmic model in this study shows an average performance improvement of 10% to 30% in global optimal path search, as well as convergence performance including loss and reward values.</div></div>","PeriodicalId":51534,"journal":{"name":"Travel Behaviour and Society","volume":"39 ","pages":"Article 100943"},"PeriodicalIF":5.1000,"publicationDate":"2024-11-05","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Travel Behaviour and Society","FirstCategoryId":"5","ListUrlMain":"https://www.sciencedirect.com/science/article/pii/S2214367X24002060","RegionNum":2,"RegionCategory":"工程技术","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"TRANSPORTATION","Score":null,"Total":0}
引用次数: 0
Abstract
Recently, Mobility as a Service (MaaS) has garnered increasing attention by integrating various modes of transportation to provide users with a unified travel solution. However, In multimodal transportation planning, we primarily face three challenges: Firstly, a multimodal travel network is constructed that covers multiple travel modes and is highly scalable. Secondly, the routing algorithm fully considers the dynamic and real-time nature of the multimodal travel process. Finally, a generalized travel cost objective function is constructed that considers the psychological burden of transfers on passengers in multimodal travel scenarios. In this study, we firstly constructed an integrated multimodal transport network based on graph theory, which covers four transport modes, namely, the metro, the bus, the car-sharing and the walking. Subsequently, by introducing a double-Q learning mechanism and an optimized dynamic exploration strategy, we propose a new algorithm, Q_EDQ, the algorithm aims to learn the globally optimal path as efficiently as possible, with faster convergence speed and improved stability. Experiments utilizing real bus and metro data from Xi’an, Shaanxi Province, were conducted to compare the Q_EDQ algorithm with traditional genetic algorithms. In the conducted four experiments, compared to the optimal paths planned by traditional genetic algorithms, the improved Q-algorithm achieved a minimum efficiency increase of 12.52% and a maximum of 35%. These results demonstrate the enhanced capability of the improved Q-algorithm to learn globally optimal paths in complex multimodal transportation networks. Compared to the classical Q algorithm, the algorithmic model in this study shows an average performance improvement of 10% to 30% in global optimal path search, as well as convergence performance including loss and reward values.
期刊介绍:
Travel Behaviour and Society is an interdisciplinary journal publishing high-quality original papers which report leading edge research in theories, methodologies and applications concerning transportation issues and challenges which involve the social and spatial dimensions. In particular, it provides a discussion forum for major research in travel behaviour, transportation infrastructure, transportation and environmental issues, mobility and social sustainability, transportation geographic information systems (TGIS), transportation and quality of life, transportation data collection and analysis, etc.