Developing an eco-driving strategy in a hybrid traffic network using reinforcement learning.

IF 2.6 4区 综合性期刊 Q2 MULTIDISCIPLINARY SCIENCES Science Progress Pub Date : 2024-07-01 DOI:10.1177/00368504241263406
Umar Jamil, Mostafa Malmir, Alan Chen, Monika Filipovska, Mimi Xie, Caiwen Ding, Yu-Fang Jin
{"title":"Developing an eco-driving strategy in a hybrid traffic network using reinforcement learning.","authors":"Umar Jamil, Mostafa Malmir, Alan Chen, Monika Filipovska, Mimi Xie, Caiwen Ding, Yu-Fang Jin","doi":"10.1177/00368504241263406","DOIUrl":null,"url":null,"abstract":"<p><p>Eco-driving has garnered considerable research attention owing to its potential socio-economic impact, including enhanced public health and mitigated climate change effects through the reduction of greenhouse gas emissions. With an expectation of more autonomous vehicles (AVs) on the road, an eco-driving strategy in hybrid traffic networks encompassing AV and human-driven vehicles (HDVs) with the coordination of traffic lights is a challenging task. The challenge is partially due to the insufficient infrastructure for collecting, transmitting, and sharing real-time traffic data among vehicles, facilities, and traffic control centers, and the following decision-making of agents involved in traffic control. Additionally, the intricate nature of the existing traffic network, with its diverse array of vehicles and facilities, contributes to the challenge by hindering the development of a mathematical model for accurately characterizing the traffic network. In this study, we utilized the Simulation of Urban Mobility (SUMO) simulator to tackle the first challenge through computational analysis. To address the second challenge, we employed a model-free reinforcement learning (RL) algorithm, proximal policy optimization, to decide the actions of AV and traffic light signals in a traffic network. A novel eco-driving strategy was proposed by introducing different percentages of AV into the traffic flow and collaborating with traffic light signals using RL to control the overall speed of the vehicles, resulting in improved fuel consumption efficiency. Average rewards with different penetration rates of AV (5%, 10%, and 20% of total vehicles) were compared to the situation without any AV in the traffic flow (0% penetration rate). The 10% penetration rate of AV showed a minimum time of convergence to achieve average reward, leading to a significant reduction in fuel consumption and total delay of all vehicles.</p>","PeriodicalId":56061,"journal":{"name":"Science Progress","volume":"107 3","pages":"368504241263406"},"PeriodicalIF":2.6000,"publicationDate":"2024-07-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC11320699/pdf/","citationCount":"0","resultStr":null,"platform":"Semanticscholar","paperid":null,"PeriodicalName":"Science Progress","FirstCategoryId":"103","ListUrlMain":"https://doi.org/10.1177/00368504241263406","RegionNum":4,"RegionCategory":"综合性期刊","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q2","JCRName":"MULTIDISCIPLINARY SCIENCES","Score":null,"Total":0}
引用次数: 0

Abstract

Eco-driving has garnered considerable research attention owing to its potential socio-economic impact, including enhanced public health and mitigated climate change effects through the reduction of greenhouse gas emissions. With an expectation of more autonomous vehicles (AVs) on the road, an eco-driving strategy in hybrid traffic networks encompassing AV and human-driven vehicles (HDVs) with the coordination of traffic lights is a challenging task. The challenge is partially due to the insufficient infrastructure for collecting, transmitting, and sharing real-time traffic data among vehicles, facilities, and traffic control centers, and the following decision-making of agents involved in traffic control. Additionally, the intricate nature of the existing traffic network, with its diverse array of vehicles and facilities, contributes to the challenge by hindering the development of a mathematical model for accurately characterizing the traffic network. In this study, we utilized the Simulation of Urban Mobility (SUMO) simulator to tackle the first challenge through computational analysis. To address the second challenge, we employed a model-free reinforcement learning (RL) algorithm, proximal policy optimization, to decide the actions of AV and traffic light signals in a traffic network. A novel eco-driving strategy was proposed by introducing different percentages of AV into the traffic flow and collaborating with traffic light signals using RL to control the overall speed of the vehicles, resulting in improved fuel consumption efficiency. Average rewards with different penetration rates of AV (5%, 10%, and 20% of total vehicles) were compared to the situation without any AV in the traffic flow (0% penetration rate). The 10% penetration rate of AV showed a minimum time of convergence to achieve average reward, leading to a significant reduction in fuel consumption and total delay of all vehicles.

查看原文
分享 分享
微信好友 朋友圈 QQ好友 复制链接
本刊更多论文
利用强化学习在混合交通网络中制定生态驾驶策略。
生态驾驶具有潜在的社会经济影响,包括通过减少温室气体排放来提高公众健康水平和减轻气候变化影响,因此受到了相当多的研究关注。随着更多自动驾驶车辆(AV)有望上路,在包括自动驾驶车辆和人类驾驶车辆(HDV)的混合交通网络中,生态驾驶战略与交通信号灯的协调是一项具有挑战性的任务。造成这一挑战的部分原因是,在车辆、设施和交通控制中心之间收集、传输和共享实时交通数据的基础设施不足,以及参与交通控制的代理决策不足。此外,现有交通网络错综复杂,车辆和设施种类繁多,阻碍了准确描述交通网络特征的数学模型的开发,从而加剧了这一挑战。在本研究中,我们利用城市交通仿真(SUMO)模拟器,通过计算分析来应对第一个挑战。为了应对第二个挑战,我们采用了一种无模型强化学习(RL)算法--近端策略优化,来决定交通网络中 AV 和交通信号灯的行动。我们提出了一种新颖的生态驾驶策略,即在交通流中引入不同比例的自动驾驶汽车,并利用 RL 与交通信号灯合作控制车辆的总体速度,从而提高燃油消耗效率。将不同普及率(占车辆总数的 5%、10% 和 20%)的自动驾驶汽车的平均回报与交通流中没有任何自动驾驶汽车的情况(普及率为 0%)进行了比较。10% 的 AV 渗透率显示,实现平均奖励的收敛时间最短,从而显著降低了所有车辆的燃油消耗和总延迟。
本文章由计算机程序翻译,如有差异,请以英文原文为准。
求助全文
约1分钟内获得全文 去求助
来源期刊
Science Progress
Science Progress Multidisciplinary-Multidisciplinary
CiteScore
3.80
自引率
0.00%
发文量
119
期刊介绍: Science Progress has for over 100 years been a highly regarded review publication in science, technology and medicine. Its objective is to excite the readers'' interest in areas with which they may not be fully familiar but which could facilitate their interest, or even activity, in a cognate field.
期刊最新文献
A voltage mode grounded capacitance multiplier with widely tunable gain for ultra-low cutoff frequency filter. Appropriate dose of tranexamic acid in the topical treatment of anterior epistaxis, 500 mg vs 1000 mg: A double-blind randomized controlled trial. Research status and prospect of flexible optimization design methodology of propeller CNC polishing machines. Sliding mode control with self-adaptive parameters of a 5-DOF hybrid robot. Spoofing attack recognition for GNSS-based train positioning using a BO-LightGBM method.
×
引用
GB/T 7714-2015
复制
MLA
复制
APA
复制
导出至
BibTeX EndNote RefMan NoteFirst NoteExpress
×
×
提示
您的信息不完整,为了账户安全,请先补充。
现在去补充
×
提示
您因"违规操作"
具体请查看互助需知
我知道了
×
提示
现在去查看 取消
×
提示
确定
0
微信
客服QQ
Book学术公众号 扫码关注我们
反馈
×
意见反馈
请填写您的意见或建议
请填写您的手机或邮箱
已复制链接
已复制链接
快去分享给好友吧!
我知道了
×
扫码分享
扫码分享
Book学术官方微信
Book学术文献互助
Book学术文献互助群
群 号:481959085
Book学术
文献互助 智能选刊 最新文献 互助须知 联系我们:info@booksci.cn
Book学术提供免费学术资源搜索服务,方便国内外学者检索中英文文献。致力于提供最便捷和优质的服务体验。
Copyright © 2023 Book学术 All rights reserved.
ghs 京公网安备 11010802042870号 京ICP备2023020795号-1