Urban traffic control is a multifaceted and demanding task that necessitates extensive decision-making to ensure the safety and efficiency of urban transportation systems. Traditional approaches require traffic signal professionals to manually intervene on traffic control devices at the intersection level, utilizing their knowledge and expertise. However, this process is cumbersome, labor-intensive, and cannot be applied on a large network scale. Recent studies have begun to explore the applicability of recommendation system for urban traffic control, which offer increased control efficiency and scalability. Such a decision recommendation system is complex, with various interdependent components, but a systematic literature review has not yet been conducted. In this work, we present an up-to-date survey that elucidates all the detailed components of a recommendation system for urban traffic control, demonstrates the utility and efficacy of such a system in the real world using data and knowledge-driven approaches, and discusses the current challenges and potential future directions of this field.
{"title":"Urban Traffic Control Meets Decision Recommendation System: A Survey and Perspective","authors":"Qingyuan Ji;Xiaoyue Wen;Junchen Jin;Yongdong Zhu;Yisheng Lv","doi":"10.1109/JAS.2024.124659","DOIUrl":"https://doi.org/10.1109/JAS.2024.124659","url":null,"abstract":"Urban traffic control is a multifaceted and demanding task that necessitates extensive decision-making to ensure the safety and efficiency of urban transportation systems. Traditional approaches require traffic signal professionals to manually intervene on traffic control devices at the intersection level, utilizing their knowledge and expertise. However, this process is cumbersome, labor-intensive, and cannot be applied on a large network scale. Recent studies have begun to explore the applicability of recommendation system for urban traffic control, which offer increased control efficiency and scalability. Such a decision recommendation system is complex, with various interdependent components, but a systematic literature review has not yet been conducted. In this work, we present an up-to-date survey that elucidates all the detailed components of a recommendation system for urban traffic control, demonstrates the utility and efficacy of such a system in the real world using data and knowledge-driven approaches, and discusses the current challenges and potential future directions of this field.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":"11 10","pages":"2043-2058"},"PeriodicalIF":15.3,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137523","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dear Editor, This letter deals with state estimation issues of discrete-time non-linear systems subject to denial-of-service (DoS) attacks under the try-once-discard (TOD) protocol. More specifically, to reduce the communication burden, a TOD protocol with novel update rules on protocol weights is designed for scheduling measurement outputs. In addition, unknown nonlinear functions vulnerable to DoS attacks are considered due to the openness and vulnerability of the network. For such systems, the neural networks (NNs) are exploited to estimate the unknown nonlinear system dynamics in the designed Luenberger-like observer. With the help of Lyapunov theory, some sufficient conditions are derived under which the estimation error and the approximation errors of NNs weights are uniformly ultimately bounded (UUB). Finally, the validity of designed observers is demonstrated by a power system example.
亲爱的编辑，这封信讨论了在 "尝试-一次-丢弃"（TOD）协议下受到拒绝服务（DoS）攻击的离散-时间非线性系统的状态估计问题。更具体地说，为了减少通信负担，我们设计了一种具有新颖的协议权重更新规则的 TOD 协议，用于调度测量输出。此外，由于网络的开放性和脆弱性，还考虑了容易受到 DoS 攻击的未知非线性函数。对于此类系统，利用神经网络（NN）来估计所设计的类似卢恩贝格尔观测器的未知非线性系统动态。在 Lyapunov 理论的帮助下，推导出了一些充分条件，在这些条件下，神经网络权重的估计误差和近似误差是均匀最终有界的（UUB）。最后，通过一个电力系统实例证明了所设计的观测器的有效性。
{"title":"Neural Network-Based State Estimation for Nonlinear Systems with Denial-of-Service Attack Under Try-Once-Discard Protocol","authors":"Xueli Wang;Shangwei Zhao;Ming Yang;Xin Wang;Xiaoming Wu","doi":"10.1109/JAS.2023.123690","DOIUrl":"https://doi.org/10.1109/JAS.2023.123690","url":null,"abstract":"Dear Editor, This letter deals with state estimation issues of discrete-time non-linear systems subject to denial-of-service (DoS) attacks under the try-once-discard (TOD) protocol. More specifically, to reduce the communication burden, a TOD protocol with novel update rules on protocol weights is designed for scheduling measurement outputs. In addition, unknown nonlinear functions vulnerable to DoS attacks are considered due to the openness and vulnerability of the network. For such systems, the neural networks (NNs) are exploited to estimate the unknown nonlinear system dynamics in the designed Luenberger-like observer. With the help of Lyapunov theory, some sufficient conditions are derived under which the estimation error and the approximation errors of NNs weights are uniformly ultimately bounded (UUB). Finally, the validity of designed observers is demonstrated by a power system example.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":"11 10","pages":"2182-2184"},"PeriodicalIF":15.3,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10664600","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137598","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dear Editor, This letter puts forward a novel scalable temporal dimension preserved tensor completion model based on orthogonal initialization for missing traffic data (MTD) imputation. The MTD imputation acts directly on accessing the traffic state, and affects the traffic management. However, it still faces the following challenges: 1) The MTD imputation is usually formulated as matrix completion or tensor completion, which ignores the information across different dimensions; 2) Most of the existing models cannot generalize to traffic datasets of different scales or different missing rates; and 3) The MTD imputation models based on Gaussian random initialization easily leads to gradient explosion or vanishing, so that the training accuracy is not effectively improved. Inspired by these findings, the proposed scalable temporal dimension preserved tensor completion (ST-DPTC) model creatively establishes the following three-fold ideas: a) Incorporating the dimension preserved tensor completion (DPTC) to extract more distinctive traffic structure changes from the low-rank latent factor tensors; b) Adopting a scalable temporal (ST) regularization with first-order difference and second-order difference operators to adapt to different scales of traffic data; and c) Embedding ST regularization into DPTC with orthogonal initialization to perform low-rank latent factor tensor extraction and MTD imputation. Results on real-world traffic datasets with different scales show that our proposed model exceeds the state-of-the-art models in terms of the imputation accuracy.
亲爱的编辑，这封信提出了一种基于正交初始化的新型可扩展时维保留张量补全模型，用于缺失交通数据（MTD）估算。MTD 估算直接作用于访问流量状态，并影响流量管理。然而，它仍面临以下挑战：1）MTD 估算通常被表述为矩阵补全或张量补全，忽略了不同维度的信息；2）现有模型大多无法泛化到不同尺度或不同缺失率的交通数据集；3）基于高斯随机初始化的 MTD 估算模型容易导致梯度爆炸或消失，从而无法有效提高训练精度。受这些发现的启发，所提出的可扩展时维保留张量补全（ST-DPTC）模型创造性地建立了以下三方面的思想：a) 结合维度保留张量补全（DPTC），从低阶潜因子张量中提取更独特的交通结构变化；b) 采用一阶差分和二阶差分算子的可扩展时间（ST）正则化，以适应不同尺度的交通数据；以及 c) 将 ST 正则化嵌入 DPTC，并进行正交初始化，以执行低阶潜因子张量提取和 MTD 估算。在不同规模的真实交通数据集上的结果表明，我们提出的模型在估算准确性方面超过了最先进的模型。
{"title":"Scalable Temporal Dimension Preserved Tensor Completion for Missing Traffic Data Imputation with Orthogonal Initialization","authors":"Hong Chen;Mingwei Lin;Jiaqi Liu;Zeshui Xu","doi":"10.1109/JAS.2024.124278","DOIUrl":"https://doi.org/10.1109/JAS.2024.124278","url":null,"abstract":"Dear Editor, This letter puts forward a novel scalable temporal dimension preserved tensor completion model based on orthogonal initialization for missing traffic data (MTD) imputation. The MTD imputation acts directly on accessing the traffic state, and affects the traffic management. However, it still faces the following challenges: 1) The MTD imputation is usually formulated as matrix completion or tensor completion, which ignores the information across different dimensions; 2) Most of the existing models cannot generalize to traffic datasets of different scales or different missing rates; and 3) The MTD imputation models based on Gaussian random initialization easily leads to gradient explosion or vanishing, so that the training accuracy is not effectively improved. Inspired by these findings, the proposed scalable temporal dimension preserved tensor completion (ST-DPTC) model creatively establishes the following three-fold ideas: a) Incorporating the dimension preserved tensor completion (DPTC) to extract more distinctive traffic structure changes from the low-rank latent factor tensors; b) Adopting a scalable temporal (ST) regularization with first-order difference and second-order difference operators to adapt to different scales of traffic data; and c) Embedding ST regularization into DPTC with orthogonal initialization to perform low-rank latent factor tensor extraction and MTD imputation. Results on real-world traffic datasets with different scales show that our proposed model exceeds the state-of-the-art models in terms of the imputation accuracy.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":"11 10","pages":"2188-2190"},"PeriodicalIF":15.3,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10664599","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137608","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Guodong Li;Bowei Zhao;Xiaorui Su;Dongxu Li;Yue Yang;Zhi Zeng;Lun Hu
N6-methyladenosine (m6A) is an important RNA methylation modification involved in regulating diverse biological processes across multiple species. Hence, the identification of m6A modification sites provides valuable insight into the biological mechanisms of complex diseases at the post-transcriptional level. Although a variety of identification algorithms have been proposed recently, most of them capture the features of m6A modification sites by focusing on the sequential dependencies of nucleotides at different positions in RNA sequences, while ignoring the structural dependencies of nucleotides in their three-dimensional structures. To overcome this issue, we propose a cross-species end-to-end deep learning model, namely CR-NSSD, which conduct a cross-domain representation learning process integrating nucleotide structural and sequential dependencies for RNA m6A site identification. Specifically, CR-NSSD first obtains the pre-coded representations of RNA sequences by incorporating the position information into single-nucleotide states with chaos game representation theory. It then constructs a cross-domain reconstruction encoder to learn the sequential and structural dependencies between nucleotides. By minimizing the reconstruction and binary cross-entropy losses, CR-NSSD is trained to complete the task of m6A site identification. Extensive experiments have demonstrated the promising performance of CR-NSSD by comparing it with several state-of-the-art m6A identification algorithms. Moreover, the results of cross-species prediction indicate that the integration of sequential and structural dependencies allows CR-NSSD to capture general features of m6A modification sites among different species, thus improving the accuracy of cross-species identification.
{"title":"Learning Sequential and Structural Dependencies Between Nucleotides for RNA N6-Methyladenosine Site Identification","authors":"Guodong Li;Bowei Zhao;Xiaorui Su;Dongxu Li;Yue Yang;Zhi Zeng;Lun Hu","doi":"10.1109/JAS.2024.124233","DOIUrl":"https://doi.org/10.1109/JAS.2024.124233","url":null,"abstract":"N6-methyladenosine (m6A) is an important RNA methylation modification involved in regulating diverse biological processes across multiple species. Hence, the identification of m6A modification sites provides valuable insight into the biological mechanisms of complex diseases at the post-transcriptional level. Although a variety of identification algorithms have been proposed recently, most of them capture the features of m6A modification sites by focusing on the sequential dependencies of nucleotides at different positions in RNA sequences, while ignoring the structural dependencies of nucleotides in their three-dimensional structures. To overcome this issue, we propose a cross-species end-to-end deep learning model, namely CR-NSSD, which conduct a cross-domain representation learning process integrating nucleotide structural and sequential dependencies for RNA m6A site identification. Specifically, CR-NSSD first obtains the pre-coded representations of RNA sequences by incorporating the position information into single-nucleotide states with chaos game representation theory. It then constructs a cross-domain reconstruction encoder to learn the sequential and structural dependencies between nucleotides. By minimizing the reconstruction and binary cross-entropy losses, CR-NSSD is trained to complete the task of m6A site identification. Extensive experiments have demonstrated the promising performance of CR-NSSD by comparing it with several state-of-the-art m6A identification algorithms. Moreover, the results of cross-species prediction indicate that the integration of sequential and structural dependencies allows CR-NSSD to capture general features of m6A modification sites among different species, thus improving the accuracy of cross-species identification.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":"11 10","pages":"2123-2134"},"PeriodicalIF":15.3,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137513","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This article studies the fault detection filtering design problem for Roesser type two-dimensional (2-D) nonlinear systems described by uncertain 2-D Takagi-Sugeno (T-S) fuzzy models. Firstly, fuzzy Lyapunov functions are constructed and the 2-D Fourier transform is exploited, based on which a finite frequency fault detection filtering design method is proposed such that a residual signal is generated with robustness to external disturbances and sensitivity to faults. It has been shown that the utilization of available frequency spectrum information of faults and disturbances makes the proposed filtering design method more general and less conservative compared with a conventional non-frequency based filtering design approach. Then, with the proposed evaluation function and its threshold, a novel mixed finite frequency $mathcal{H}_{infty}/mathcal{H}_{-}$ fault detection algorithm is developed, based on which the fault can be immediately detected once the evaluation function exceeds the threshold. Finally, it is verified with simulation studies that the proposed method is effective and less conservative than conventional non-frequency and/or common Lyapunov function based filtering design methods.
{"title":"Fuzzy-Model-Based Finite Frequency Fault Detection Filtering Design for Two-Dimensional Nonlinear Systems","authors":"Meng Wang;Huaicheng Yan;Jianbin Qiu;Wenqiang Ji","doi":"10.1109/JAS.2024.124452","DOIUrl":"https://doi.org/10.1109/JAS.2024.124452","url":null,"abstract":"This article studies the fault detection filtering design problem for Roesser type two-dimensional (2-D) nonlinear systems described by uncertain 2-D Takagi-Sugeno (T-S) fuzzy models. Firstly, fuzzy Lyapunov functions are constructed and the 2-D Fourier transform is exploited, based on which a finite frequency fault detection filtering design method is proposed such that a residual signal is generated with robustness to external disturbances and sensitivity to faults. It has been shown that the utilization of available frequency spectrum information of faults and disturbances makes the proposed filtering design method more general and less conservative compared with a conventional non-frequency based filtering design approach. Then, with the proposed evaluation function and its threshold, a novel mixed finite frequency \u0000<tex>$mathcal{H}_{infty}/mathcal{H}_{-}$</tex>\u0000 fault detection algorithm is developed, based on which the fault can be immediately detected once the evaluation function exceeds the threshold. Finally, it is verified with simulation studies that the proposed method is effective and less conservative than conventional non-frequency and/or common Lyapunov function based filtering design methods.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":"11 10","pages":"2099-2110"},"PeriodicalIF":15.3,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137530","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Shouyong Jiang;Jinglei Guo;Yong Wang;Shengxiang Yang
Decomposition of a complex multi-objective optimisation problem (MOP) to multiple simple subMOPs, known as M2M for short, is an effective approach to multi-objective optimisation. However, M2M facilitates little communication/collaboration between subMOPs, which limits its use in complex optimisation scenarios. This paper extends the M2M framework to develop a unified algorithm for both multi-objective and many-objective optimisation. Through bilevel decomposition, an MOP is divided into multiple subMOPs at upper level, each of which is further divided into a number of single-objective subproblems at lower level. Neighbouring subMOPs are allowed to share some subproblems so that the knowledge gained from solving one sub-MOP can be transferred to another, and eventually to all the sub-MOPs. The bilevel decomposition is readily combined with some new mating selection and population update strategies, leading to a high-performance algorithm that competes effectively against a number of state-of-the-arts studied in this paper for both multi- and many-objective optimisation. Parameter analysis and component analysis have been also carried out to further justify the proposed algorithm.
{"title":"Evolutionary Multi/Many-Objective Optimisation via Bilevel Decomposition","authors":"Shouyong Jiang;Jinglei Guo;Yong Wang;Shengxiang Yang","doi":"10.1109/JAS.2024.124515","DOIUrl":"https://doi.org/10.1109/JAS.2024.124515","url":null,"abstract":"Decomposition of a complex multi-objective optimisation problem (MOP) to multiple simple subMOPs, known as M2M for short, is an effective approach to multi-objective optimisation. However, M2M facilitates little communication/collaboration between subMOPs, which limits its use in complex optimisation scenarios. This paper extends the M2M framework to develop a unified algorithm for both multi-objective and many-objective optimisation. Through bilevel decomposition, an MOP is divided into multiple subMOPs at upper level, each of which is further divided into a number of single-objective subproblems at lower level. Neighbouring subMOPs are allowed to share some subproblems so that the knowledge gained from solving one sub-MOP can be transferred to another, and eventually to all the sub-MOPs. The bilevel decomposition is readily combined with some new mating selection and population update strategies, leading to a high-performance algorithm that competes effectively against a number of state-of-the-arts studied in this paper for both multi- and many-objective optimisation. Parameter analysis and component analysis have been also carried out to further justify the proposed algorithm.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":"11 9","pages":"1973-1986"},"PeriodicalIF":15.3,"publicationDate":"2024-08-16","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141993916","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper presents a risk-informed data-driven safe control design approach for a class of stochastic uncertain nonlinear discrete-time systems. The nonlinear system is modeled using linear parameter-varying (LPV) systems. A model-based probabilistic safe controller is first designed to guarantee probabilistic $lambda$ -contractivity (i.e., stability and invariance) of the LPV system with respect to a given polyhedral safe set. To obviate the requirement of knowing the LPV system model and to bypass identifying its open-loop model, its closed-loop data-based representation is provided in terms of state and scheduling data as well as a decision variable. It is shown that the variance of the closed-loop system, as well as the probability of safety satisfaction, depends on the decision variable and the noise covariance. A minimum-variance direct data-driven gain-scheduling safe control design approach is presented next by designing the decision variable such that all possible closed-loop system realizations satisfy safety with the highest confidence level. This minimum-variance approach is a control-oriented learning method since it minimizes the variance of the state of the closed-loop system with respect to the safe set, and thus minimizes the risk of safety violation. Unlike the certainty-equivalent approach that results in a risk-neutral control design, the minimum-variance method leads to a risk-averse control design. It is shown that the presented direct risk-averse learning approach requires weaker data richness conditions than existing indirect learning methods based on system identification and can lead to a lower risk of safety violation. Two simulation examples along with an experimental validation on an autonomous vehicle are provided to show the effectiveness of the presented approach.
{"title":"Risk-Informed Model-Free Safe Control of Linear Parameter-Varying Systems","authors":"Babak Esmaeili;Hamidreza Modares","doi":"10.1109/JAS.2024.124479","DOIUrl":"https://doi.org/10.1109/JAS.2024.124479","url":null,"abstract":"This paper presents a risk-informed data-driven safe control design approach for a class of stochastic uncertain nonlinear discrete-time systems. The nonlinear system is modeled using linear parameter-varying (LPV) systems. A model-based probabilistic safe controller is first designed to guarantee probabilistic \u0000<tex>$lambda$</tex>\u0000-contractivity (i.e., stability and invariance) of the LPV system with respect to a given polyhedral safe set. To obviate the requirement of knowing the LPV system model and to bypass identifying its open-loop model, its closed-loop data-based representation is provided in terms of state and scheduling data as well as a decision variable. It is shown that the variance of the closed-loop system, as well as the probability of safety satisfaction, depends on the decision variable and the noise covariance. A minimum-variance direct data-driven gain-scheduling safe control design approach is presented next by designing the decision variable such that all possible closed-loop system realizations satisfy safety with the highest confidence level. This minimum-variance approach is a control-oriented learning method since it minimizes the variance of the state of the closed-loop system with respect to the safe set, and thus minimizes the risk of safety violation. Unlike the certainty-equivalent approach that results in a risk-neutral control design, the minimum-variance method leads to a risk-averse control design. It is shown that the presented direct risk-averse learning approach requires weaker data richness conditions than existing indirect learning methods based on system identification and can lead to a lower risk of safety violation. Two simulation examples along with an experimental validation on an autonomous vehicle are provided to show the effectiveness of the presented approach.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":"11 9","pages":"1918-1932"},"PeriodicalIF":15.3,"publicationDate":"2024-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141991489","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This article studies the effective traffic signal control problem of multiple intersections in a city-level traffic system. A novel regional multi-agent cooperative reinforcement learning algorithm called RegionSTLight is proposed to improve the traffic efficiency. Firstly a regional multi-agent Q-learning framework is proposed, which can equivalently decompose the global Q value of the traffic system into the local values of several regions. Based on the framework and the idea of human-machine cooperation, a dynamic zoning method is designed to divide the traffic network into several strong-coupled regions according to real-time traffic flow densities. In order to achieve better cooperation inside each region, a lightweight spatio-temporal fusion feature extraction network is designed. The experiments in synthetic, real-world and city-level scenarios show that the proposed RegionSTLight converges more quickly, is more stable, and obtains better asymptotic performance compared to state-of-the-art models.
{"title":"Regional Multi-Agent Cooperative Reinforcement Learning for City-Level Traffic Grid Signal Control","authors":"Yisha Li;Ya Zhang;Xinde Li;Changyin Sun","doi":"10.1109/JAS.2024.124365","DOIUrl":"https://doi.org/10.1109/JAS.2024.124365","url":null,"abstract":"This article studies the effective traffic signal control problem of multiple intersections in a city-level traffic system. A novel regional multi-agent cooperative reinforcement learning algorithm called RegionSTLight is proposed to improve the traffic efficiency. Firstly a regional multi-agent Q-learning framework is proposed, which can equivalently decompose the global Q value of the traffic system into the local values of several regions. Based on the framework and the idea of human-machine cooperation, a dynamic zoning method is designed to divide the traffic network into several strong-coupled regions according to real-time traffic flow densities. In order to achieve better cooperation inside each region, a lightweight spatio-temporal fusion feature extraction network is designed. The experiments in synthetic, real-world and city-level scenarios show that the proposed RegionSTLight converges more quickly, is more stable, and obtains better asymptotic performance compared to state-of-the-art models.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":"11 9","pages":"1987-1998"},"PeriodicalIF":15.3,"publicationDate":"2024-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141991509","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
This paper studies the target controllability of multi-layer complex networked systems, in which the nodes are high-dimensional linear time invariant (LTI) dynamical systems, and the network topology is directed and weighted. The influence of inter-layer couplings on the target controllability of multi-layer networks is discussed. It is found that even if there exists a layer which is not target controllable, the entire multi-layer network can still be target controllable due to the inter-layer couplings. For the multi-layer networks with general structure, a necessary and sufficient condition for target controllability is given by establishing the relationship between uncontrollable subspace and output matrix. By the derived condition, it can be found that the system may be target controllable even if it is not state controllable. On this basis, two corollaries are derived, which clarify the relationship between target controllability, state controllability and output controllability. For the multi-layer networks where the inter-layer couplings are directed chains and directed stars, sufficient conditions for target controllability of networked systems are given, respectively. These conditions are easier to verify than the classic criterion.
{"title":"Target Controllability of Multi-Layer Networks with High-Dimensional Nodes","authors":"Lifu Wang;Zhaofei Li;Ge Guo;Zhi Kong","doi":"10.1109/JAS.2023.124152","DOIUrl":"https://doi.org/10.1109/JAS.2023.124152","url":null,"abstract":"This paper studies the target controllability of multi-layer complex networked systems, in which the nodes are high-dimensional linear time invariant (LTI) dynamical systems, and the network topology is directed and weighted. The influence of inter-layer couplings on the target controllability of multi-layer networks is discussed. It is found that even if there exists a layer which is not target controllable, the entire multi-layer network can still be target controllable due to the inter-layer couplings. For the multi-layer networks with general structure, a necessary and sufficient condition for target controllability is given by establishing the relationship between uncontrollable subspace and output matrix. By the derived condition, it can be found that the system may be target controllable even if it is not state controllable. On this basis, two corollaries are derived, which clarify the relationship between target controllability, state controllability and output controllability. For the multi-layer networks where the inter-layer couplings are directed chains and directed stars, sufficient conditions for target controllability of networked systems are given, respectively. These conditions are easier to verify than the classic criterion.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":"11 9","pages":"1999-2010"},"PeriodicalIF":15.3,"publicationDate":"2024-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141991492","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Unsignalized intersections pose a challenge for autonomous vehicles that must decide how to navigate them safely and efficiently. This paper proposes a reinforcement learning (RL) method for autonomous vehicles to navigate unsignalized intersections safely and efficiently. The method uses a semantic scene representation to handle variable numbers of vehicles and a universal reward function to facilitate stable learning. A collision risk function is designed to penalize unsafe actions and guide the agent to avoid them. A scalable policy optimization algorithm is introduced to improve data efficiency and safety for vehicle learning at intersections. The algorithm employs experience replay to overcome the on-policy limitation of proximal policy optimization and incorporates the collision risk constraint into the policy optimization problem. The proposed safe RL algorithm can balance the trade-off between vehicle traffic safety and policy learning efficiency. Simulated intersection scenarios with different traffic situations are used to test the algorithm and demonstrate its high success rates and low collision rates under different traffic conditions. The algorithm shows the potential of RL for enhancing the safety and reliability of autonomous driving systems at unsignalized intersections.
{"title":"Safe Efficient Policy Optimization Algorithm for Unsignalized Intersection Navigation","authors":"Xiaolong Chen;Biao Xu;Manjiang Hu;Yougang Bian;Yang Li;Xin Xu","doi":"10.1109/JAS.2024.124287","DOIUrl":"https://doi.org/10.1109/JAS.2024.124287","url":null,"abstract":"Unsignalized intersections pose a challenge for autonomous vehicles that must decide how to navigate them safely and efficiently. This paper proposes a reinforcement learning (RL) method for autonomous vehicles to navigate unsignalized intersections safely and efficiently. The method uses a semantic scene representation to handle variable numbers of vehicles and a universal reward function to facilitate stable learning. A collision risk function is designed to penalize unsafe actions and guide the agent to avoid them. A scalable policy optimization algorithm is introduced to improve data efficiency and safety for vehicle learning at intersections. The algorithm employs experience replay to overcome the on-policy limitation of proximal policy optimization and incorporates the collision risk constraint into the policy optimization problem. The proposed safe RL algorithm can balance the trade-off between vehicle traffic safety and policy learning efficiency. Simulated intersection scenarios with different traffic situations are used to test the algorithm and demonstrate its high success rates and low collision rates under different traffic conditions. The algorithm shows the potential of RL for enhancing the safety and reliability of autonomous driving systems at unsignalized intersections.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":"11 9","pages":"2011-2026"},"PeriodicalIF":15.3,"publicationDate":"2024-08-15","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"141991473","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：