Pub Date : 2022-06-02DOI: 10.1007/s43684-022-00028-0
Andrew E. Bondoc, Mohsen Tayefeh, Ahmad Barari
Digital Twins are essential in establishing intelligent asset management for an asset or machine. They can be described as the bidirectional communication between a cyber representation and a physical asset. Predictive Maintenance is dependent on the existence of three data sets: Fault history, Maintenance/Repair History, and Machine Conditions. Current Digital Twin solutions can fail to simulate the behaviour of a faulty asset. These solutions also prove to be difficult to implement when an asset’s fault history is incomplete. This paper presents the novel methodology, LIVE Digital Twin, to develop Digital Twins with the focus of Predictive Maintenance. The four phases, Learn, Identify, Verify, and Extend are discussed. A case study analyzes the relationship of component stiffness and vibration in detecting the health of various components. The Learning phase is implemented to demonstrate the process of locating a preliminary sensor network and develop the faulty history of a Sand Removal Skid assembly. Future studies will consider fewer simplifying assumptions and expand on the results to implement the proceeding phases.
数字孪生系统对建立资产或机器的智能资产管理至关重要。它们可以被描述为网络表征和物理资产之间的双向通信。预测性维护依赖于三个数据集的存在:故障历史、维护/维修历史和机器状况。当前的数字孪生解决方案可能无法模拟故障资产的行为。当资产的故障历史记录不完整时,这些解决方案也很难实施。本文介绍了一种名为 LIVE Digital Twin 的新方法,用于开发以预测性维护为重点的数字孪生系统。本文讨论了学习、识别、验证和扩展四个阶段。案例研究分析了组件刚度和振动在检测各种组件健康状况中的关系。学习阶段用于演示初步传感器网络的定位过程,以及开发除沙橇组件的故障历史。未来的研究将考虑减少简化假设,并在结果的基础上进一步实施后续阶段。
{"title":"Learning phase in a LIVE Digital Twin for predictive maintenance","authors":"Andrew E. Bondoc, Mohsen Tayefeh, Ahmad Barari","doi":"10.1007/s43684-022-00028-0","DOIUrl":"10.1007/s43684-022-00028-0","url":null,"abstract":"<div><p>Digital Twins are essential in establishing intelligent asset management for an asset or machine. They can be described as the bidirectional communication between a cyber representation and a physical asset. Predictive Maintenance is dependent on the existence of three data sets: <i>Fault history</i>, <i>Maintenance</i>/<i>Repair History</i>, and <i>Machine Conditions</i>. Current Digital Twin solutions can fail to simulate the behaviour of a faulty asset. These solutions also prove to be difficult to implement when an asset’s fault history is incomplete. This paper presents the novel methodology, LIVE Digital Twin, to develop Digital Twins with the focus of Predictive Maintenance. The four phases, Learn, Identify, Verify, and Extend are discussed. A case study analyzes the relationship of component stiffness and vibration in detecting the health of various components. The Learning phase is implemented to demonstrate the process of locating a preliminary sensor network and develop the faulty history of a Sand Removal Skid assembly. Future studies will consider fewer simplifying assumptions and expand on the results to implement the proceeding phases.</p></div>","PeriodicalId":71187,"journal":{"name":"自主智能系统(英文)","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-06-02","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s43684-022-00028-0.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48505202","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-06-01DOI: 10.1007/s43684-022-00031-5
Kaijie Lu, Chong Chen, Tao Wang, Lianglun Cheng, Jian Qin
Fault diagnosis plays a vital role in assessing the health management of industrial robots and improving maintenance schedules. In recent decades, artificial intelligence-based data-driven approaches have made significant progress in machine fault diagnosis using monitoring data. However, current methods pay less attention to correlations and internal differences in monitoring data, resulting in limited diagnostic performance. In this paper, a data-driven method is proposed for the fault diagnosis of industrial robot reducers, that is, a dual-module attention convolutional neural network (DMA-CNN). This method aims to diagnose the fault state of industrial robot reducer. It establishes two parallel convolutional neural networks with two different attentions to capture the different features related to the fault. Finally, the features are fused to obtain the fault diagnosis results (normal or abnormal). The fault diagnosis effect of the DMA-CNN method and other attention models are compared and analyzed. The effectiveness of the method is verified on a dataset of real industrial robots.
{"title":"Fault diagnosis of industrial robot based on dual-module attention convolutional neural network","authors":"Kaijie Lu, Chong Chen, Tao Wang, Lianglun Cheng, Jian Qin","doi":"10.1007/s43684-022-00031-5","DOIUrl":"10.1007/s43684-022-00031-5","url":null,"abstract":"<div><p>Fault diagnosis plays a vital role in assessing the health management of industrial robots and improving maintenance schedules. In recent decades, artificial intelligence-based data-driven approaches have made significant progress in machine fault diagnosis using monitoring data. However, current methods pay less attention to correlations and internal differences in monitoring data, resulting in limited diagnostic performance. In this paper, a data-driven method is proposed for the fault diagnosis of industrial robot reducers, that is, a dual-module attention convolutional neural network (DMA-CNN). This method aims to diagnose the fault state of industrial robot reducer. It establishes two parallel convolutional neural networks with two different attentions to capture the different features related to the fault. Finally, the features are fused to obtain the fault diagnosis results (normal or abnormal). The fault diagnosis effect of the DMA-CNN method and other attention models are compared and analyzed. The effectiveness of the method is verified on a dataset of real industrial robots.</p></div>","PeriodicalId":71187,"journal":{"name":"自主智能系统(英文)","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-06-01","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s43684-022-00031-5.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48044082","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-05-31DOI: 10.1007/s43684-022-00030-6
Yuxin Liu, Chong Chen, Tao Wang, Lianglun Cheng
An industrial robot is a complex mechatronics system, whose failure is hard to diagnose based on monitoring data. Previous studies have reported various methods with deep network models to improve the accuracy of fault diagnosis, which can get an accurate prediction model when the amount of data sample is sufficient. However, the failure data is hard to obtain, which leads to the few-shot issue and the bad generalization ability of the model. Therefore, this paper proposes an attention enhanced dilated convolutional neural network (D-CNN) approach for the cross-axis industrial robotics fault diagnosis method. Firstly, key feature extraction and sliding window are adopted to pre-process the monitoring data of industrial robots before D-CNN is introduced to extract data features. And self-attention is used to enhance feature attention capability. Finally, the pre-trained model is used for transfer learning, and a small number of the dataset from another axis of the multi-axis industrial robot are used for fine-tuning experiments. The experimental results show that the proposed method can reach satisfactory fault diagnosis accuracy in both the source domain and target domain.
{"title":"An attention enhanced dilated CNN approach for cross-axis industrial robotics fault diagnosis","authors":"Yuxin Liu, Chong Chen, Tao Wang, Lianglun Cheng","doi":"10.1007/s43684-022-00030-6","DOIUrl":"10.1007/s43684-022-00030-6","url":null,"abstract":"<div><p>An industrial robot is a complex mechatronics system, whose failure is hard to diagnose based on monitoring data. Previous studies have reported various methods with deep network models to improve the accuracy of fault diagnosis, which can get an accurate prediction model when the amount of data sample is sufficient. However, the failure data is hard to obtain, which leads to the few-shot issue and the bad generalization ability of the model. Therefore, this paper proposes an attention enhanced dilated convolutional neural network (D-CNN) approach for the cross-axis industrial robotics fault diagnosis method. Firstly, key feature extraction and sliding window are adopted to pre-process the monitoring data of industrial robots before D-CNN is introduced to extract data features. And self-attention is used to enhance feature attention capability. Finally, the pre-trained model is used for transfer learning, and a small number of the dataset from another axis of the multi-axis industrial robot are used for fine-tuning experiments. The experimental results show that the proposed method can reach satisfactory fault diagnosis accuracy in both the source domain and target domain.</p></div>","PeriodicalId":71187,"journal":{"name":"自主智能系统(英文)","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-05-31","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s43684-022-00030-6.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"46725199","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-05-27DOI: 10.1007/s43684-022-00027-1
Yanqiong Zhang, Chaoqun Liu, Yu-Ping Tian
In this paper, the constrained Nash equilibrium seeking problem of aggregative games is investigated for uncertain nonlinear Euler-Lagrange (EL) systems under unbalanced digraphs, where the cost function for each agent depends on its own decision variable and the aggregate of all other decisions. By embedding a distributed estimator of the left eigenvector associated with zero eigenvalue of the digraph Laplacian matrix, a dynamic adaptive average consensus protocol is employed to estimate the aggregate function in the unbalanced case. To solve the constrained Nash equilibrium seeking problem, an integrated distributed protocol based on output-constrained nonlinear control and projected dynamics is proposed for uncertain EL players to reach the Nash equilibrium. The convergence analysis is established by using variational inequality technique and Lyapunov stability analysis. Finally, a numerical example in electricity market is provided to validate the effectiveness of the proposed method.
本文研究了不平衡数字图下不确定非线性欧拉-拉格朗日(EL)系统的聚合博弈受限纳什均衡寻求问题,其中每个代理的成本函数取决于其自身的决策变量和所有其他决策的总和。通过嵌入与数图拉普拉奇矩阵零特征值相关的左特征向量的分布式估计器,采用动态自适应平均共识协议来估计不平衡情况下的合计函数。为解决受限纳什均衡寻求问题,提出了一种基于输出受限非线性控制和投影动力学的集成分布式协议,用于不确定的 EL 参与者达到纳什均衡。利用变分不等式技术和 Lyapunov 稳定性分析建立了收敛性分析。最后,以电力市场为例,验证了所提方法的有效性。
{"title":"Distributed constrained aggregative games of uncertain Euler-Lagrange systems under unbalanced digraphs","authors":"Yanqiong Zhang, Chaoqun Liu, Yu-Ping Tian","doi":"10.1007/s43684-022-00027-1","DOIUrl":"10.1007/s43684-022-00027-1","url":null,"abstract":"<div><p>In this paper, the constrained Nash equilibrium seeking problem of aggregative games is investigated for uncertain nonlinear Euler-Lagrange (EL) systems under unbalanced digraphs, where the cost function for each agent depends on its own decision variable and the aggregate of all other decisions. By embedding a distributed estimator of the left eigenvector associated with zero eigenvalue of the digraph Laplacian matrix, a dynamic adaptive average consensus protocol is employed to estimate the aggregate function in the unbalanced case. To solve the constrained Nash equilibrium seeking problem, an integrated distributed protocol based on output-constrained nonlinear control and projected dynamics is proposed for uncertain EL players to reach the Nash equilibrium. The convergence analysis is established by using variational inequality technique and Lyapunov stability analysis. Finally, a numerical example in electricity market is provided to validate the effectiveness of the proposed method.</p></div>","PeriodicalId":71187,"journal":{"name":"自主智能系统(英文)","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-05-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s43684-022-00027-1.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"48421696","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-05-27DOI: 10.1007/s43684-022-00029-z
Yuki Miyashita, Toshiharu Sugawara
We propose a two-stage reward allocation method with decay using an extension of replay memory to adapt this rewarding method for deep reinforcement learning (DRL), to generate coordinated behaviors for tasks that can be completed by executing a few subtasks sequentially by heterogeneous agents. An independent learner in cooperative multi-agent systems needs to learn its policies for effective execution of its own responsible subtask, as well as for coordinated behaviors under a certain coordination structure. Although the reward scheme is an issue for DRL, it is difficult to design it to learn both policies. Our proposed method attempts to generate these different behaviors in multi-agent DRL by dividing the timing of rewards into two stages and varying the ratio between them over time. By introducing the coordinated delivery and execution problem with an expiration time, where a task can be executed sequentially by two heterogeneous agents, we experimentally analyze the effect of using various ratios of the reward division in the two-stage allocations on the generated behaviors. The results demonstrate that the proposed method could improve the overall performance relative to those with the conventional one-time or fixed reward and can establish robust coordinated behavior.
{"title":"Two-stage reward allocation with decay for multi-agent coordinated behavior for sequential cooperative task by using deep reinforcement learning","authors":"Yuki Miyashita, Toshiharu Sugawara","doi":"10.1007/s43684-022-00029-z","DOIUrl":"10.1007/s43684-022-00029-z","url":null,"abstract":"<div><p>We propose a two-stage reward allocation method with decay using an extension of replay memory to adapt this rewarding method for deep reinforcement learning (DRL), to generate coordinated behaviors for tasks that can be completed by executing a few subtasks sequentially by heterogeneous agents. An independent learner in cooperative multi-agent systems needs to learn its policies for effective execution of its own responsible subtask, as well as for coordinated behaviors under a certain coordination structure. Although the reward scheme is an issue for DRL, it is difficult to design it to learn both policies. Our proposed method attempts to generate these different behaviors in multi-agent DRL by dividing the timing of rewards into two stages and varying the ratio between them over time. By introducing the coordinated delivery and execution problem with an expiration time, where a task can be executed sequentially by two heterogeneous agents, we experimentally analyze the effect of using various ratios of the reward division in the two-stage allocations on the generated behaviors. The results demonstrate that the proposed method could improve the overall performance relative to those with the conventional one-time or fixed reward and can establish robust coordinated behavior.</p></div>","PeriodicalId":71187,"journal":{"name":"自主智能系统(英文)","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-05-27","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s43684-022-00029-z.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"45075022","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-04-29DOI: 10.1007/s43684-022-00025-3
Michael O. Macaulay, Mahmood Shafiee
Machine learning and in particular deep learning techniques have demonstrated the most efficacy in training, learning, analyzing, and modelling large complex structured and unstructured datasets. These techniques have recently been commonly deployed in different industries to support robotic and autonomous system (RAS) requirements and applications ranging from planning and navigation to machine vision and robot manipulation in complex environments. This paper reviews the state-of-the-art with regard to RAS technologies (including unmanned marine robot systems, unmanned ground robot systems, climbing and crawler robots, unmanned aerial vehicles, and space robot systems) and their application for the inspection and monitoring of mechanical systems and civil infrastructure. We explore various types of data provided by such systems and the analytical techniques being adopted to process and analyze these data. This paper provides a brief overview of machine learning and deep learning techniques, and more importantly, a classification of the literature which have reported the deployment of such techniques for RAS-based inspection and monitoring of utility pipelines, wind turbines, aircrafts, power lines, pressure vessels, bridges, etc. Our research provides documented information on the use of advanced data-driven technologies in the analysis of critical assets and examines the main challenges to the applications of such technologies in the industry.
{"title":"Machine learning techniques for robotic and autonomous inspection of mechanical systems and civil infrastructure","authors":"Michael O. Macaulay, Mahmood Shafiee","doi":"10.1007/s43684-022-00025-3","DOIUrl":"10.1007/s43684-022-00025-3","url":null,"abstract":"<div><p>Machine learning and in particular <i>deep learning</i> techniques have demonstrated the most efficacy in training, learning, analyzing, and modelling large complex structured and unstructured datasets. These techniques have recently been commonly deployed in different industries to support robotic and autonomous system (RAS) requirements and applications ranging from planning and navigation to machine vision and robot manipulation in complex environments. This paper reviews the state-of-the-art with regard to RAS technologies (including unmanned marine robot systems, unmanned ground robot systems, climbing and crawler robots, unmanned aerial vehicles, and space robot systems) and their application for the inspection and monitoring of mechanical systems and civil infrastructure. We explore various types of data provided by such systems and the analytical techniques being adopted to process and analyze these data. This paper provides a brief overview of machine learning and deep learning techniques, and more importantly, a classification of the literature which have reported the deployment of such techniques for RAS-based inspection and monitoring of utility pipelines, wind turbines, aircrafts, power lines, pressure vessels, bridges, etc. Our research provides documented information on the use of advanced data-driven technologies in the analysis of critical assets and examines the main challenges to the applications of such technologies in the industry.</p></div>","PeriodicalId":71187,"journal":{"name":"自主智能系统(英文)","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-04-29","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s43684-022-00025-3.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"42816843","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-04-18DOI: 10.1007/s43684-022-00026-2
Yutao Tang, Peng Yi, Yanqiong Zhang, Dawei Liu
In this paper, we aim to develop distributed continuous-time algorithms over directed graphs to seek the Nash equilibrium in a noncooperative game. Motivated by the recent consensus-based designs, we present a distributed algorithm with a proportional gain for weight-balanced directed graphs. By further embedding a distributed estimator of the left eigenvector associated with zero eigenvalue of the graph Laplacian, we extend it to the case with arbitrary strongly connected directed graphs having possible unbalanced weights. In both cases, the Nash equilibrium is proven to be exactly reached with an exponential convergence rate. An example is given to illustrate the validity of the theoretical results.
{"title":"Nash equilibrium seeking over directed graphs","authors":"Yutao Tang, Peng Yi, Yanqiong Zhang, Dawei Liu","doi":"10.1007/s43684-022-00026-2","DOIUrl":"10.1007/s43684-022-00026-2","url":null,"abstract":"<div><p>In this paper, we aim to develop distributed continuous-time algorithms over directed graphs to seek the Nash equilibrium in a noncooperative game. Motivated by the recent consensus-based designs, we present a distributed algorithm with a proportional gain for weight-balanced directed graphs. By further embedding a distributed estimator of the left eigenvector associated with zero eigenvalue of the graph Laplacian, we extend it to the case with arbitrary strongly connected directed graphs having possible unbalanced weights. In both cases, the Nash equilibrium is proven to be exactly reached with an exponential convergence rate. An example is given to illustrate the validity of the theoretical results.</p></div>","PeriodicalId":71187,"journal":{"name":"自主智能系统(英文)","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-04-18","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s43684-022-00026-2.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43002504","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-04-12DOI: 10.1007/s43684-022-00024-4
Shu Liang, Peng Yi, Yiguang Hong, Kaixiang Peng
Distributed Nash equilibrium seeking of aggregative games is investigated and a continuous-time algorithm is proposed. The algorithm is designed by virtue of projected gradient play dynamics and aggregation tracking dynamics, and is applicable to games with constrained strategy sets and weight-balanced communication graphs. The key feature of our method is that the proposed projected dynamics achieves exponential convergence, whereas such convergence results are only obtained for non-projected dynamics in existing works on distributed optimization and equilibrium seeking. Numerical examples illustrate the effectiveness of our methods.
{"title":"Exponentially convergent distributed Nash equilibrium seeking for constrained aggregative games","authors":"Shu Liang, Peng Yi, Yiguang Hong, Kaixiang Peng","doi":"10.1007/s43684-022-00024-4","DOIUrl":"10.1007/s43684-022-00024-4","url":null,"abstract":"<div><p>Distributed Nash equilibrium seeking of aggregative games is investigated and a continuous-time algorithm is proposed. The algorithm is designed by virtue of projected gradient play dynamics and aggregation tracking dynamics, and is applicable to games with constrained strategy sets and weight-balanced communication graphs. The key feature of our method is that the proposed projected dynamics achieves exponential convergence, whereas such convergence results are only obtained for non-projected dynamics in existing works on distributed optimization and equilibrium seeking. Numerical examples illustrate the effectiveness of our methods.</p></div>","PeriodicalId":71187,"journal":{"name":"自主智能系统(英文)","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-04-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s43684-022-00024-4.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"49479824","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-03-16DOI: 10.1007/s43684-022-00023-5
Wei Zhou, Dong Chen, Jun Yan, Zhaojian Li, Huilin Yin, Wanchen Ge
Autonomous driving has attracted significant research interests in the past two decades as it offers many potential benefits, including releasing drivers from exhausting driving and mitigating traffic congestion, among others. Despite promising progress, lane-changing remains a great challenge for autonomous vehicles (AV), especially in mixed and dynamic traffic scenarios. Recently, reinforcement learning (RL) has been widely explored for lane-changing decision makings in AVs with encouraging results demonstrated. However, the majority of those studies are focused on a single-vehicle setting, and lane-changing in the context of multiple AVs coexisting with human-driven vehicles (HDVs) have received scarce attention. In this paper, we formulate the lane-changing decision-making of multiple AVs in a mixed-traffic highway environment as a multi-agent reinforcement learning (MARL) problem, where each AV makes lane-changing decisions based on the motions of both neighboring AVs and HDVs. Specifically, a multi-agent advantage actor-critic (MA2C) method is proposed with a novel local reward design and a parameter sharing scheme. In particular, a multi-objective reward function is designed to incorporate fuel efficiency, driving comfort, and the safety of autonomous driving. A comprehensive experimental study is made that our proposed MARL framework consistently outperforms several state-of-the-art benchmarks in terms of efficiency, safety, and driver comfort.
{"title":"Multi-agent reinforcement learning for cooperative lane changing of connected and autonomous vehicles in mixed traffic","authors":"Wei Zhou, Dong Chen, Jun Yan, Zhaojian Li, Huilin Yin, Wanchen Ge","doi":"10.1007/s43684-022-00023-5","DOIUrl":"10.1007/s43684-022-00023-5","url":null,"abstract":"<div><p>Autonomous driving has attracted significant research interests in the past two decades as it offers many potential benefits, including releasing drivers from exhausting driving and mitigating traffic congestion, among others. Despite promising progress, lane-changing remains a great challenge for autonomous vehicles (AV), especially in mixed and dynamic traffic scenarios. Recently, reinforcement learning (RL) has been widely explored for lane-changing decision makings in AVs with encouraging results demonstrated. However, the majority of those studies are focused on a single-vehicle setting, and lane-changing in the context of multiple AVs coexisting with human-driven vehicles (HDVs) have received scarce attention. In this paper, we formulate the lane-changing decision-making of multiple AVs in a mixed-traffic highway environment as a multi-agent reinforcement learning (MARL) problem, where each AV makes lane-changing decisions based on the motions of both neighboring AVs and HDVs. Specifically, a multi-agent advantage actor-critic (MA2C) method is proposed with a novel local reward design and a parameter sharing scheme. In particular, a multi-objective reward function is designed to incorporate fuel efficiency, driving comfort, and the safety of autonomous driving. A comprehensive experimental study is made that our proposed MARL framework consistently outperforms several state-of-the-art benchmarks in terms of efficiency, safety, and driver comfort.</p></div>","PeriodicalId":71187,"journal":{"name":"自主智能系统(英文)","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-03-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s43684-022-00023-5.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"52856324","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Pub Date : 2022-03-12DOI: 10.1007/s43684-022-00022-6
Gautier Vanson, Pascale Marangé, Eric Levrat
Circular economy enables to restore product value at the end of life i.e. when no longer used or damaged. Thus, the product life cycle is extended and this economy permits to reduce waste increase and resources rarefaction. There are several revaluation options (reuse, remanufacturing, recycling, …). So, decision makers need to assess these options to determine which is the best decision. Thus, we will present a study about an End-Of-Life (EoL) decision making which aims to facilitate the industrialization of circular economy. For this, it is essential to consider all variables and parameters impacting the decision of the product trajectory. A first part of the work proposes to identify the variables and parameters impacting the decision making. A second part proposes an assessment approach based on a modeling by Generalized Colored Stochastic Petri Net (GCSPN) and on a Monte-Carlo simulation. The approach developed is tested on an industrial example from the literature to analyze the efficiency and effectiveness of the model. This first application showed the feasibility of the approach, and also the limits of the GCSPN modelling.
循环经济能够在产品寿命终止时,即不再使用或损坏时,恢复产品的价值。这样,产品的生命周期就得到了延长,这种经济可以减少废物的增加和资源的稀缺。有几种重估选择(再利用、再制造、再循环......)。因此,决策者需要对这些方案进行评估,以确定哪个是最佳决策。因此,我们将介绍一项关于寿命终结(EoL)决策的研究,旨在促进循环经济的产业化。为此,必须考虑影响产品轨迹决策的所有变量和参数。工作的第一部分建议确定影响决策的变量和参数。第二部分提出了一种基于广义彩色随机 Petri 网(GCSPN)建模和蒙特卡洛模拟的评估方法。该方法在文献中的一个工业实例中进行了测试,以分析模型的效率和有效性。首次应用表明了该方法的可行性,以及 GCSPN 建模的局限性。
{"title":"End-of-Life Decision making in circular economy using generalized colored stochastic Petri nets","authors":"Gautier Vanson, Pascale Marangé, Eric Levrat","doi":"10.1007/s43684-022-00022-6","DOIUrl":"10.1007/s43684-022-00022-6","url":null,"abstract":"<div><p>Circular economy enables to restore product value at the end of life i.e. when no longer used or damaged. Thus, the product life cycle is extended and this economy permits to reduce waste increase and resources rarefaction. There are several revaluation options (reuse, remanufacturing, recycling, …). So, decision makers need to assess these options to determine which is the best decision. Thus, we will present a study about an End-Of-Life (EoL) decision making which aims to facilitate the industrialization of circular economy. For this, it is essential to consider all variables and parameters impacting the decision of the product trajectory. A first part of the work proposes to identify the variables and parameters impacting the decision making. A second part proposes an assessment approach based on a modeling by Generalized Colored Stochastic Petri Net (GCSPN) and on a Monte-Carlo simulation. The approach developed is tested on an industrial example from the literature to analyze the efficiency and effectiveness of the model. This first application showed the feasibility of the approach, and also the limits of the GCSPN modelling.</p></div>","PeriodicalId":71187,"journal":{"name":"自主智能系统(英文)","volume":"2 1","pages":""},"PeriodicalIF":0.0,"publicationDate":"2022-03-12","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://link.springer.com/content/pdf/10.1007/s43684-022-00022-6.pdf","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"43164694","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":0,"RegionCategory":"","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}