Huaguang Zhang;Lulu Zhang;Jiayue Sun;Tianbiao Wang
{"title":"通过自适应动态编程实现具有半马尔可夫跳跃参数的未知非线性系统的最优控制","authors":"Huaguang Zhang;Lulu Zhang;Jiayue Sun;Tianbiao Wang","doi":"10.1109/TSMC.2024.3421658","DOIUrl":null,"url":null,"abstract":"This article investigates the optimal control problem for the discrete-time (DT) nonlinear semi-Markovian jump systems (s-MJSs) that possess unknown dynamics. The study uses the semi-Markovian kernel approach to address the problem of mode-switching in these systems. This approach employs the transition probability and the sojourn-time distribution function to jointly determine the transitions between different modes. Then, with a neural network (NN) identifier, the demand for accurate information on the system dynamics is eliminated, and an optimal control method for the nonlinear s-MJSs is utilized to solve the Hamilton-Jacobi–Bellman equation (HJBE) built upon adaptive dynamic programming methodology. Additionally, a detailed analysis of the convergence of a value iteration-based algorithm, which solves the optimal control issue for the DT s-MJSs, is thoroughly discussed. Furthermore, an actor-critic NN is trained to attain an estimated solution to the relevant HJBE. Finally, to validate the designed approach, two simulations are performed to prove its effectiveness.","PeriodicalId":48915,"journal":{"name":"IEEE Transactions on Systems Man Cybernetics-Systems","volume":null,"pages":null},"PeriodicalIF":8.6000,"publicationDate":"2024-07-23","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Optimal Control for Unknown Nonlinear System With Semi-Markovian Jump Parameters via Adaptive Dynamic Programming\",\"authors\":\"Huaguang Zhang;Lulu Zhang;Jiayue Sun;Tianbiao Wang\",\"doi\":\"10.1109/TSMC.2024.3421658\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"This article investigates the optimal control problem for the discrete-time (DT) nonlinear semi-Markovian jump systems (s-MJSs) that possess unknown dynamics. The study uses the semi-Markovian kernel approach to address the problem of mode-switching in these systems. This approach employs the transition probability and the sojourn-time distribution function to jointly determine the transitions between different modes. Then, with a neural network (NN) identifier, the demand for accurate information on the system dynamics is eliminated, and an optimal control method for the nonlinear s-MJSs is utilized to solve the Hamilton-Jacobi–Bellman equation (HJBE) built upon adaptive dynamic programming methodology. Additionally, a detailed analysis of the convergence of a value iteration-based algorithm, which solves the optimal control issue for the DT s-MJSs, is thoroughly discussed. Furthermore, an actor-critic NN is trained to attain an estimated solution to the relevant HJBE. Finally, to validate the designed approach, two simulations are performed to prove its effectiveness.\",\"PeriodicalId\":48915,\"journal\":{\"name\":\"IEEE Transactions on Systems Man Cybernetics-Systems\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":8.6000,\"publicationDate\":\"2024-07-23\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"IEEE Transactions on Systems Man Cybernetics-Systems\",\"FirstCategoryId\":\"94\",\"ListUrlMain\":\"https://ieeexplore.ieee.org/document/10607965/\",\"RegionNum\":1,\"RegionCategory\":\"计算机科学\",\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q1\",\"JCRName\":\"AUTOMATION & CONTROL SYSTEMS\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"IEEE Transactions on Systems Man Cybernetics-Systems","FirstCategoryId":"94","ListUrlMain":"https://ieeexplore.ieee.org/document/10607965/","RegionNum":1,"RegionCategory":"计算机科学","ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q1","JCRName":"AUTOMATION & CONTROL SYSTEMS","Score":null,"Total":0}
Optimal Control for Unknown Nonlinear System With Semi-Markovian Jump Parameters via Adaptive Dynamic Programming
This article investigates the optimal control problem for the discrete-time (DT) nonlinear semi-Markovian jump systems (s-MJSs) that possess unknown dynamics. The study uses the semi-Markovian kernel approach to address the problem of mode-switching in these systems. This approach employs the transition probability and the sojourn-time distribution function to jointly determine the transitions between different modes. Then, with a neural network (NN) identifier, the demand for accurate information on the system dynamics is eliminated, and an optimal control method for the nonlinear s-MJSs is utilized to solve the Hamilton-Jacobi–Bellman equation (HJBE) built upon adaptive dynamic programming methodology. Additionally, a detailed analysis of the convergence of a value iteration-based algorithm, which solves the optimal control issue for the DT s-MJSs, is thoroughly discussed. Furthermore, an actor-critic NN is trained to attain an estimated solution to the relevant HJBE. Finally, to validate the designed approach, two simulations are performed to prove its effectiveness.
期刊介绍:
The IEEE Transactions on Systems, Man, and Cybernetics: Systems encompasses the fields of systems engineering, covering issue formulation, analysis, and modeling throughout the systems engineering lifecycle phases. It addresses decision-making, issue interpretation, systems management, processes, and various methods such as optimization, modeling, and simulation in the development and deployment of large systems.