Dear Editor, This letter focuses on the collision avoidance for a multi-unmanned surface vehicle (multi-USV) system. A novel multi-USV collision avoidance (MUCA) algorithm is proposed. Firstly, in order to get a more reasonable collision avoidance policy, reward functions are constructed according to international regulations for preventing col-lisions at sea (COLREGS) and USV dynamics. Secondly, to reduce data noises and the impacts of outliers, an improved normalization method is proposed. States and rewards of USVs are normalized to avoid gradient vanishing and exploding. Thirdly, a novel $epsilon$ -greedy method is proposed to help the optimal policy converge faster. It is easier for USVs to explore the optimal policy in the learning process. Finally, the proposed MUCA algorithm is tested in a multi-encounter situation including head-on, crossing, and overtaking. The experimental results demonstrate that the newly proposed MUCA algorithm can provide a collision-free marching policy for the USVs in formation.
{"title":"Multi-USV Formation Collision Avoidance via Deep Reinforcement Learning and COLREGs","authors":"Cheng-Cheng Wang;Yu-Long Wang;Li Jia","doi":"10.1109/JAS.2023.123846","DOIUrl":"https://doi.org/10.1109/JAS.2023.123846","url":null,"abstract":"Dear Editor, This letter focuses on the collision avoidance for a multi-unmanned surface vehicle (multi-USV) system. A novel multi-USV collision avoidance (MUCA) algorithm is proposed. Firstly, in order to get a more reasonable collision avoidance policy, reward functions are constructed according to international regulations for preventing col-lisions at sea (COLREGS) and USV dynamics. Secondly, to reduce data noises and the impacts of outliers, an improved normalization method is proposed. States and rewards of USVs are normalized to avoid gradient vanishing and exploding. Thirdly, a novel \u0000<tex>$epsilon$</tex>\u0000-greedy method is proposed to help the optimal policy converge faster. It is easier for USVs to explore the optimal policy in the learning process. Finally, the proposed MUCA algorithm is tested in a multi-encounter situation including head-on, crossing, and overtaking. The experimental results demonstrate that the newly proposed MUCA algorithm can provide a collision-free marching policy for the USVs in formation.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10707649","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142397479","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Zhongyuan Zhao;Zhiqiang Yang;Luyao Jiang;Ju Yang;Quanbo Ge
This paper considers the distributed online optimization (DOO) problem over time-varying unbalanced networks, where gradient information is explicitly unknown. To address this issue, a privacy-preserving distributed online one-point residual feedback (OPRF) optimization algorithm is proposed. This algorithm updates decision variables by leveraging one-point residual feedback to estimate the true gradient information. It can achieve the same performance as the two-point feedback scheme while only requiring a single function value query per iteration. Additionally, it effectively eliminates the effect of time-varying unbalanced graphs by dynamically constructing row stochastic matrices. Furthermore, compared to other distributed optimization algorithms that only consider explicitly unknown cost functions, this paper also addresses the issue of privacy information leakage of nodes. Theoretical analysis demonstrate that the method attains sublinear regret while protecting the privacy information of agents. Finally, numerical experiments on distributed collaborative localization problem and federated learning confirm the effectiveness of the algorithm.
{"title":"Privacy Preserving Distributed Bandit Residual Feedback Online Optimization Over Time-Varying Unbalanced Graphs","authors":"Zhongyuan Zhao;Zhiqiang Yang;Luyao Jiang;Ju Yang;Quanbo Ge","doi":"10.1109/JAS.2024.124656","DOIUrl":"https://doi.org/10.1109/JAS.2024.124656","url":null,"abstract":"This paper considers the distributed online optimization (DOO) problem over time-varying unbalanced networks, where gradient information is explicitly unknown. To address this issue, a privacy-preserving distributed online one-point residual feedback (OPRF) optimization algorithm is proposed. This algorithm updates decision variables by leveraging one-point residual feedback to estimate the true gradient information. It can achieve the same performance as the two-point feedback scheme while only requiring a single function value query per iteration. Additionally, it effectively eliminates the effect of time-varying unbalanced graphs by dynamically constructing row stochastic matrices. Furthermore, compared to other distributed optimization algorithms that only consider explicitly unknown cost functions, this paper also addresses the issue of privacy information leakage of nodes. Theoretical analysis demonstrate that the method attains sublinear regret while protecting the privacy information of agents. Finally, numerical experiments on distributed collaborative localization problem and federated learning confirm the effectiveness of the algorithm.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142397368","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dear Editor, Recently, with the development of artificial intelligence, game intelligence decision-making has attracted more and more attention. In particular, incomplete-information games (IIG) have gradually become a new research focus, where players make decisions without sufficient information, such as the opponent's strategies or preferences. In this case, a selfish player can only make reactive decisions based on the changes in environment and state. Thus, blind decisions by players may drift them away from the path of reward maximization, and may even hinder the health of the IIG environment. Therefore, it is necessary to design an effective mechanism to optimize decision-making for IIG players.
{"title":"A Linear Programming-Based Reinforcement Learning Mechanism for Incomplete-Information Games","authors":"Baosen Yang;Changbing Tang;Yang Liu;Guanghui Wen;Guanrong Chen","doi":"10.1109/JAS.2024.124464","DOIUrl":"https://doi.org/10.1109/JAS.2024.124464","url":null,"abstract":"Dear Editor, Recently, with the development of artificial intelligence, game intelligence decision-making has attracted more and more attention. In particular, incomplete-information games (IIG) have gradually become a new research focus, where players make decisions without sufficient information, such as the opponent's strategies or preferences. In this case, a selfish player can only make reactive decisions based on the changes in environment and state. Thus, blind decisions by players may drift them away from the path of reward maximization, and may even hinder the health of the IIG environment. Therefore, it is necessary to design an effective mechanism to optimize decision-making for IIG players.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10707689","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142397478","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
In this paper, we study stealthy cyber-attacks on actuators of cyber-physical systems (CPS), namely zero dynamics and controllable attacks. In particular, under certain assumptions, we investigate and propose conditions under which one can execute zero dynamics and controllable attacks in the CPS. The above conditions are derived based on the Markov parameters of the CPS and elements of the system observability matrix. Consequently, in addition to outlining the number of required actuators to be attacked, these conditions provide one with the minimum system knowledge needed to perform zero dynamics and controllable cyber-attacks. As a countermeasure against the above stealthy cyber-attacks, we develop a dynamic coding scheme that increases the minimum number of the CPS required actuators to carry out zero dynamics and controllable cyber-attacks to its maximum possible value. It is shown that if at least one secure input channel exists, the proposed dynamic coding scheme can prevent adversaries from executing the zero dynamics and controllable attacks even if they have complete knowledge of the coding system. Finally, two illustrative numerical case studies are provided to demonstrate the effectiveness and capabilities of our derived conditions and proposed methodologies.
{"title":"On Zero Dynamics and Controllable Cyber-Attacks in Cyber-Physical Systems and Dynamic Coding Schemes as Their Countermeasures","authors":"Mahdi Taheri;Khashayar Khorasani;Nader Meskin","doi":"10.1109/JAS.2024.124692","DOIUrl":"https://doi.org/10.1109/JAS.2024.124692","url":null,"abstract":"In this paper, we study stealthy cyber-attacks on actuators of cyber-physical systems (CPS), namely zero dynamics and controllable attacks. In particular, under certain assumptions, we investigate and propose conditions under which one can execute zero dynamics and controllable attacks in the CPS. The above conditions are derived based on the Markov parameters of the CPS and elements of the system observability matrix. Consequently, in addition to outlining the number of required actuators to be attacked, these conditions provide one with the minimum system knowledge needed to perform zero dynamics and controllable cyber-attacks. As a countermeasure against the above stealthy cyber-attacks, we develop a dynamic coding scheme that increases the minimum number of the CPS required actuators to carry out zero dynamics and controllable cyber-attacks to its maximum possible value. It is shown that if at least one secure input channel exists, the proposed dynamic coding scheme can prevent adversaries from executing the zero dynamics and controllable attacks even if they have complete knowledge of the coding system. Finally, two illustrative numerical case studies are provided to demonstrate the effectiveness and capabilities of our derived conditions and proposed methodologies.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-10-08","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142397143","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
For the $n$ -qubit stochastic open quantum systems, based on the Lyapunov stability theorem and LaSalle's invariant set principle, a pure state switching control based on on-line estimated state feedback (short for OQST-SFC) is proposed to realize the state transition the pure state of the target state including eigenstate and superposition state. The proposed switching control consists of a constant control and a control law designed based on the Lyapunov method, in which the Lyapunov function is the state distance of the system. The constant control is used to drive the system state from an initial state to the convergence domain only containing the target state, and a Lyapunov-based control is used to make the state enter the convergence domain and then continue to converge to the target state. At the same time, the continuous weak measurement of quantum system and the quantum state tomography method based on the on-line alternating direction multiplier (QST-OADM) are used to obtain the system information and estimate the quantum state which is used as the input of the quantum system controller. Then, the pure state feedback switching control method based on the on-line estimated state feedback is realized in an $n$ -qubit stochastic open quantum system. The complete derivation process of $n$ -qubit QST-OADM algorithm is given; Through strict theoretical proof and analysis, the convergence conditions to ensure any initial state of the quantum system to converge the target pure state are given. The proposed control method is applied to a 2-qubit stochastic open quantum system for numerical simulation experiments. Four possible different position cases between the initial estimated state and that of the controlled system are studied and discussed, and the performances of the state transition under the corresponding cases are analyzed.
{"title":"Pure State Feedback Switching Control Based on the Online Estimated State for Stochastic Open Quantum Systems","authors":"Shuang Cong;Zhixiang Dong","doi":"10.1109/JAS.2023.124071","DOIUrl":"https://doi.org/10.1109/JAS.2023.124071","url":null,"abstract":"For the \u0000<tex>$n$</tex>\u0000-qubit stochastic open quantum systems, based on the Lyapunov stability theorem and LaSalle's invariant set principle, a pure state switching control based on on-line estimated state feedback (short for OQST-SFC) is proposed to realize the state transition the pure state of the target state including eigenstate and superposition state. The proposed switching control consists of a constant control and a control law designed based on the Lyapunov method, in which the Lyapunov function is the state distance of the system. The constant control is used to drive the system state from an initial state to the convergence domain only containing the target state, and a Lyapunov-based control is used to make the state enter the convergence domain and then continue to converge to the target state. At the same time, the continuous weak measurement of quantum system and the quantum state tomography method based on the on-line alternating direction multiplier (QST-OADM) are used to obtain the system information and estimate the quantum state which is used as the input of the quantum system controller. Then, the pure state feedback switching control method based on the on-line estimated state feedback is realized in an \u0000<tex>$n$</tex>\u0000-qubit stochastic open quantum system. The complete derivation process of \u0000<tex>$n$</tex>\u0000-qubit QST-OADM algorithm is given; Through strict theoretical proof and analysis, the convergence conditions to ensure any initial state of the quantum system to converge the target pure state are given. The proposed control method is applied to a 2-qubit stochastic open quantum system for numerical simulation experiments. Four possible different position cases between the initial estimated state and that of the controlled system are studied and discussed, and the performances of the state transition under the corresponding cases are analyzed.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137610","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Dear Editor, This letter addresses the predefined-time control for cooperative tracking of multiple quadrotor unmanned aerial vehicles (UAVs) under a directed communication network. A predefined-time distributed estimator is first introduced to accurately get the reference velocity and acceleration for each UAV. Then, a cascade predefined-time control strategy is proposed to guarantee that all the UAVs track the reference trajectory while maintaining a preassigned configuration, where an attitude constraint algorithm is developed to avoid the flipping over of each VAV. Stability analysis demonstrates that the tracking errors of the closed-loop systems converge to zero within a predefined time. Finally, experiment results validate the proposed control strategy.
{"title":"Distributed Predefined-Time Control for Cooperative Tracking of Multiple Quadrotor UAVs","authors":"Kewei Xia;Xinyi Li;Kaidan Li;Yao Zou","doi":"10.1109/JAS.2023.123861","DOIUrl":"https://doi.org/10.1109/JAS.2023.123861","url":null,"abstract":"Dear Editor, This letter addresses the predefined-time control for cooperative tracking of multiple quadrotor unmanned aerial vehicles (UAVs) under a directed communication network. A predefined-time distributed estimator is first introduced to accurately get the reference velocity and acceleration for each UAV. Then, a cascade predefined-time control strategy is proposed to guarantee that all the UAVs track the reference trajectory while maintaining a preassigned configuration, where an attitude constraint algorithm is developed to avoid the flipping over of each VAV. Stability analysis demonstrates that the tracking errors of the closed-loop systems converge to zero within a predefined time. Finally, experiment results validate the proposed control strategy.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10664603","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137529","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Linear temporal logic (LTL) is an intuitive and expressive language to specify complex control tasks, and how to design an efficient control strategy for LTL specification is still a challenge. In this paper, we implement the dynamic quantization technique to propose a novel hierarchical control strategy for nonlinear control systems under LTL specifications. Based on the regions of interest involved in the LTL formula, an accepting path is derived first to provide a high-level solution for the controller synthesis problem. Second, we develop a dynamic quantization based approach to verify the realization of the accepting path. The realization verification results in the necessity of the controller design and a sequence of quantization regions for the controller design. Third, the techniques of dynamic quantization and abstraction-based control are combined together to establish the local-to-global control strategy. Both abstraction construction and controller design are local and dynamic, thereby resulting in the potential reduction of the computational complexity. Since each quantization region can be considered locally and individually, the proposed hierarchical mechanism is more efficient and can solve much larger problems than many existing methods. Finally, the proposed control strategy is illustrated via two examples from the path planning and tracking problems of mobile robots.
{"title":"Hierarchical Controller Synthesis Under Linear Temporal Logic Specifications Using Dynamic Quantization","authors":"Wei Ren;Zhuo-Rui Pan;Weiguo Xia;Xi-Ming Sun","doi":"10.1109/JAS.2024.124473","DOIUrl":"https://doi.org/10.1109/JAS.2024.124473","url":null,"abstract":"Linear temporal logic (LTL) is an intuitive and expressive language to specify complex control tasks, and how to design an efficient control strategy for LTL specification is still a challenge. In this paper, we implement the dynamic quantization technique to propose a novel hierarchical control strategy for nonlinear control systems under LTL specifications. Based on the regions of interest involved in the LTL formula, an accepting path is derived first to provide a high-level solution for the controller synthesis problem. Second, we develop a dynamic quantization based approach to verify the realization of the accepting path. The realization verification results in the necessity of the controller design and a sequence of quantization regions for the controller design. Third, the techniques of dynamic quantization and abstraction-based control are combined together to establish the local-to-global control strategy. Both abstraction construction and controller design are local and dynamic, thereby resulting in the potential reduction of the computational complexity. Since each quantization region can be considered locally and individually, the proposed hierarchical mechanism is more efficient and can solve much larger problems than many existing methods. Finally, the proposed control strategy is illustrated via two examples from the path planning and tracking problems of mobile robots.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137597","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
To perform well, deep learning (DL) models have to be trained well. Which optimizer should be adopted? We answer this question by discussing how optimizers have evolved from traditional methods like gradient descent to more advanced techniques to address challenges posed by high-dimensional and non-convex problem space. Ongoing challenges include their hyperparameter sensitivity, balancing between convergence and generalization performance, and improving interpretability of optimization processes. Researchers continue to seek robust, efficient, and universally applicable optimizers to advance the field of DL across various domains.
{"title":"Evolution and Role of Optimizers in Training Deep Learning Models","authors":"XiaoHao Wen;MengChu Zhou","doi":"10.1109/JAS.2024.124806","DOIUrl":"https://doi.org/10.1109/JAS.2024.124806","url":null,"abstract":"To perform well, deep learning (DL) models have to be trained well. Which optimizer should be adopted? We answer this question by discussing how optimizers have evolved from traditional methods like gradient descent to more advanced techniques to address challenges posed by high-dimensional and non-convex problem space. Ongoing challenges include their hyperparameter sensitivity, balancing between convergence and generalization performance, and improving interpretability of optimization processes. Researchers continue to seek robust, efficient, and universally applicable optimizers to advance the field of DL across various domains.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=10664602","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137514","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"OA","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Intelligent machinery fault diagnosis methods have been popularly and successfully developed in the past decades, and the vibration acceleration data collected by contact accelerometers have been widely investigated. In many industrial scenarios, contactless sensors are more preferred. The event camera is an emerging bio-inspired technology for vision sensing, which asynchronously records per-pixel brightness change polarity with high temporal resolution and low latency. It offers a promising tool for contactless machine vibration sensing and fault diagnosis. However, the dynamic vision-based methods suffer from variations of practical factors such as camera position, machine operating condition, etc. Furthermore, as a new sensing technology, the labeled dynamic vision data are limited, which generally cannot cover a wide range of machine fault modes. Aiming at these challenges, a novel dynamic vision-based machinery fault diagnosis method is proposed in this paper. It is motivated to explore the abundant vibration acceleration data for enhancing the dynamic vision-based model performance. A cross-modality feature alignment method is thus proposed with deep adversarial neural networks to achieve fault diagnosis knowledge transfer. An event erasing method is further proposed for improving model robustness against variations. The proposed method can effectively identify unseen fault mode with dynamic vision data. Experiments on two rotating machine monitoring datasets are carried out for validations, and the results suggest the proposed method is promising for generalized contactless machinery fault diagnosis.
{"title":"Dynamic Vision-Based Machinery Fault Diagnosis with Cross-Modality Feature Alignment","authors":"Xiang Li;Shupeng Yu;Yaguo Lei;Naipeng Li;Bin Yang","doi":"10.1109/JAS.2024.124470","DOIUrl":"https://doi.org/10.1109/JAS.2024.124470","url":null,"abstract":"Intelligent machinery fault diagnosis methods have been popularly and successfully developed in the past decades, and the vibration acceleration data collected by contact accelerometers have been widely investigated. In many industrial scenarios, contactless sensors are more preferred. The event camera is an emerging bio-inspired technology for vision sensing, which asynchronously records per-pixel brightness change polarity with high temporal resolution and low latency. It offers a promising tool for contactless machine vibration sensing and fault diagnosis. However, the dynamic vision-based methods suffer from variations of practical factors such as camera position, machine operating condition, etc. Furthermore, as a new sensing technology, the labeled dynamic vision data are limited, which generally cannot cover a wide range of machine fault modes. Aiming at these challenges, a novel dynamic vision-based machinery fault diagnosis method is proposed in this paper. It is motivated to explore the abundant vibration acceleration data for enhancing the dynamic vision-based model performance. A cross-modality feature alignment method is thus proposed with deep adversarial neural networks to achieve fault diagnosis knowledge transfer. An event erasing method is further proposed for improving model robustness against variations. The proposed method can effectively identify unseen fault mode with dynamic vision data. Experiments on two rotating machine monitoring datasets are carried out for validations, and the results suggest the proposed method is promising for generalized contactless machinery fault diagnosis.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137595","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
Contract Bridge, a four-player imperfect information game, comprises two phases: bidding and playing. While computer programs excel at playing, bidding presents a challenging aspect due to the need for information exchange with partners and interference with communication of opponents. In this work, we introduce a Bridge bidding agent that combines supervised learning, deep reinforcement learning via self-play, and a test-time search approach. Our experiments demonstrate that our agent outperforms WBridge5, a highly regarded computer Bridge software that has won multiple world championships, by a performance of 0.98 IMPs (international match points) per deal over 10 000 deals, with a much cost-effective approach. The performance significantly surpasses previous state-of-the-art (0.85 IMPs per deal). Note 0.1 IMPs per deal is a significant improvement in Bridge bidding.
{"title":"Bridge Bidding via Deep Reinforcement Learning and Belief Monte Carlo Search","authors":"Zizhang Qiu;Shouguang Wang;Dan You;MengChu Zhou","doi":"10.1109/JAS.2024.124488","DOIUrl":"https://doi.org/10.1109/JAS.2024.124488","url":null,"abstract":"Contract Bridge, a four-player imperfect information game, comprises two phases: bidding and playing. While computer programs excel at playing, bidding presents a challenging aspect due to the need for information exchange with partners and interference with communication of opponents. In this work, we introduce a Bridge bidding agent that combines supervised learning, deep reinforcement learning via self-play, and a test-time search approach. Our experiments demonstrate that our agent outperforms WBridge5, a highly regarded computer Bridge software that has won multiple world championships, by a performance of 0.98 IMPs (international match points) per deal over 10 000 deals, with a much cost-effective approach. The performance significantly surpasses previous state-of-the-art (0.85 IMPs per deal). Note 0.1 IMPs per deal is a significant improvement in Bridge bidding.","PeriodicalId":54230,"journal":{"name":"Ieee-Caa Journal of Automatica Sinica","volume":null,"pages":null},"PeriodicalIF":15.3,"publicationDate":"2024-09-04","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":null,"resultStr":null,"platform":"Semanticscholar","paperid":"142137609","PeriodicalName":null,"FirstCategoryId":null,"ListUrlMain":null,"RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":"","EPubDate":null,"PubModel":null,"JCR":null,"JCRName":null,"Score":null,"Total":0}
京公网安备 11010802042870号
京ICP备2023020795号-1
扫码关注我们
求助内容：
应助结果提醒方式：