深度调度:基于深度强化学习的先进空中交通车辆调度算法

Q2 Social Sciences Journal of Air Transportation Pub Date : 2024-06-10 DOI:10.2514/1.d0416
Elaheh Sabziyan Varnousfaderani, S. Shihab, E. F. Dulia
{"title":"深度调度:基于深度强化学习的先进空中交通车辆调度算法","authors":"Elaheh Sabziyan Varnousfaderani, S. Shihab, E. F. Dulia","doi":"10.2514/1.d0416","DOIUrl":null,"url":null,"abstract":"Near-future air taxi operations with electric vertical takeoff and landing aircraft will be constrained by the need for frequent recharging and limited takeoff and landing pads in vertiports and will be subject to time-varying demand and electricity prices, making the dispatch problem unique and particularly challenging to solve. Previously, the authors have developed optimization models to address this problem. Such optimization models, however, suffer from prohibitively high computational run times when the scale of the problem increases, making them less practical for real-world implementation. To overcome this issue, the authors have developed two deep reinforcement learning-based dispatch algorithms, namely, single-agent and multi-agent double dueling deep Q-network dispatch algorithms, where the objective is to maximize operating profit. A passenger transportation simulation environment was built to assess the performance of these algorithms across 36 numerical cases with varying numbers of vehicles and vertiports and amounts of demand. The results indicate that the multi-agent dispatch algorithm can closely approximate the optimal dispatch policy with significantly less computational expenses compared to the benchmark optimization model. The multi-agent algorithm was found to outperform the single-agent counterpart with respect to both profits generated and training time. Additionally, we implemented a heuristic-based algorithm, faster but less effective in generating profits compared to our two deep reinforcement learning-based algorithms.","PeriodicalId":36984,"journal":{"name":"Journal of Air Transportation","volume":null,"pages":null},"PeriodicalIF":0.0000,"publicationDate":"2024-06-10","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"DeepDispatch: Deep Reinforcement Learning-Based Vehicle Dispatch Algorithm for Advanced Air Mobility\",\"authors\":\"Elaheh Sabziyan Varnousfaderani, S. Shihab, E. F. Dulia\",\"doi\":\"10.2514/1.d0416\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Near-future air taxi operations with electric vertical takeoff and landing aircraft will be constrained by the need for frequent recharging and limited takeoff and landing pads in vertiports and will be subject to time-varying demand and electricity prices, making the dispatch problem unique and particularly challenging to solve. Previously, the authors have developed optimization models to address this problem. Such optimization models, however, suffer from prohibitively high computational run times when the scale of the problem increases, making them less practical for real-world implementation. To overcome this issue, the authors have developed two deep reinforcement learning-based dispatch algorithms, namely, single-agent and multi-agent double dueling deep Q-network dispatch algorithms, where the objective is to maximize operating profit. A passenger transportation simulation environment was built to assess the performance of these algorithms across 36 numerical cases with varying numbers of vehicles and vertiports and amounts of demand. The results indicate that the multi-agent dispatch algorithm can closely approximate the optimal dispatch policy with significantly less computational expenses compared to the benchmark optimization model. The multi-agent algorithm was found to outperform the single-agent counterpart with respect to both profits generated and training time. Additionally, we implemented a heuristic-based algorithm, faster but less effective in generating profits compared to our two deep reinforcement learning-based algorithms.\",\"PeriodicalId\":36984,\"journal\":{\"name\":\"Journal of Air Transportation\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.0000,\"publicationDate\":\"2024-06-10\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Journal of Air Transportation\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.2514/1.d0416\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q2\",\"JCRName\":\"Social Sciences\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Journal of Air Transportation","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.2514/1.d0416","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"Social Sciences","Score":null,"Total":0}
引用次数: 0

摘要

在不久的将来,电动垂直起降飞机的空中出租车运营将受到频繁充电的需求和机场有限的起降坪的限制,并将受到随时间变化的需求和电价的影响,这使得调度问题非常独特,解决起来特别具有挑战性。在此之前,作者已经开发了优化模型来解决这一问题。然而,当问题规模增大时,此类优化模型的计算运行时间过高,使其在实际应用中不那么实用。为了克服这一问题,作者开发了两种基于深度强化学习的调度算法,即单代理和多代理双决斗深度 Q 网络调度算法,其目标是实现运营利润最大化。作者建立了一个客运模拟环境,以评估这些算法在 36 个具有不同车辆数量、vertiports 和需求量的数字案例中的性能。结果表明,与基准优化模型相比,多代理调度算法能以更少的计算费用接近最优调度策略。我们发现,多代理算法在产生的利润和训练时间方面都优于单代理算法。此外,我们还实施了一种基于启发式的算法,与我们的两种基于深度强化学习的算法相比,该算法速度更快,但在产生利润方面效果较差。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
DeepDispatch: Deep Reinforcement Learning-Based Vehicle Dispatch Algorithm for Advanced Air Mobility
Near-future air taxi operations with electric vertical takeoff and landing aircraft will be constrained by the need for frequent recharging and limited takeoff and landing pads in vertiports and will be subject to time-varying demand and electricity prices, making the dispatch problem unique and particularly challenging to solve. Previously, the authors have developed optimization models to address this problem. Such optimization models, however, suffer from prohibitively high computational run times when the scale of the problem increases, making them less practical for real-world implementation. To overcome this issue, the authors have developed two deep reinforcement learning-based dispatch algorithms, namely, single-agent and multi-agent double dueling deep Q-network dispatch algorithms, where the objective is to maximize operating profit. A passenger transportation simulation environment was built to assess the performance of these algorithms across 36 numerical cases with varying numbers of vehicles and vertiports and amounts of demand. The results indicate that the multi-agent dispatch algorithm can closely approximate the optimal dispatch policy with significantly less computational expenses compared to the benchmark optimization model. The multi-agent algorithm was found to outperform the single-agent counterpart with respect to both profits generated and training time. Additionally, we implemented a heuristic-based algorithm, faster but less effective in generating profits compared to our two deep reinforcement learning-based algorithms.
求助全文
通过发布文献求助,成功后即可免费获取论文全文。 去求助
来源期刊
Journal of Air Transportation
Journal of Air Transportation Social Sciences-Safety Research
CiteScore
2.80
自引率
0.00%
发文量
16
期刊最新文献
Airside Surveillance by Computer Vision in Low-Visibility and Low-Fidelity Environment Strategic Planning of Aerial Assets for Disaster Response Predicted Trajectory Accuracy Requirements to Reduce Aviation Impact of Space Launch Operations Aircraft Takeoff and Landing Weight Estimation from Surveillance Data Simulating Integration of Urban Air Mobility into Existing Transportation Systems: Survey
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1