{"title":"混合状态空间上马尔可夫链演化的稳态概率计算","authors":"Az-eddine Zakrad, A. Nasroallah","doi":"10.1515/mcma-2023-2003","DOIUrl":null,"url":null,"abstract":"Abstract The partitioning algorithm is an iterative procedure that computes explicitly the steady-state probability of a finite Markov chain 𝑋. In this paper, we propose to adapt this algorithm to the case where the state space E := C ∪ D E:=C\\cup D is composed of a continuous part 𝐶 and a finite part 𝐷 such that C ∩ D = ∅ C\\cap D=\\emptyset . In this case, the steady-state probability 𝜋 of 𝑋 is a convex combination of two steady-state probabilities π C \\pi_{C} and π D \\pi_{D} of two Markov chains on 𝐶 and 𝐷 respectively. The obtained algorithm allows to compute explicitly π D \\pi_{D} . If π C \\pi_{C} cannot be computed explicitly, our algorithm approximates it by numerical resolution of successive integral equations. Some numerical examples are studied to show the usefulness and proper functioning of our proposal.","PeriodicalId":46576,"journal":{"name":"Monte Carlo Methods and Applications","volume":null,"pages":null},"PeriodicalIF":0.8000,"publicationDate":"2023-03-30","publicationTypes":"Journal Article","fieldsOfStudy":null,"isOpenAccess":false,"openAccessPdf":"","citationCount":"0","resultStr":"{\"title\":\"Computation of the steady-state probability of Markov chain evolving on a mixed state space\",\"authors\":\"Az-eddine Zakrad, A. Nasroallah\",\"doi\":\"10.1515/mcma-2023-2003\",\"DOIUrl\":null,\"url\":null,\"abstract\":\"Abstract The partitioning algorithm is an iterative procedure that computes explicitly the steady-state probability of a finite Markov chain 𝑋. In this paper, we propose to adapt this algorithm to the case where the state space E := C ∪ D E:=C\\\\cup D is composed of a continuous part 𝐶 and a finite part 𝐷 such that C ∩ D = ∅ C\\\\cap D=\\\\emptyset . In this case, the steady-state probability 𝜋 of 𝑋 is a convex combination of two steady-state probabilities π C \\\\pi_{C} and π D \\\\pi_{D} of two Markov chains on 𝐶 and 𝐷 respectively. The obtained algorithm allows to compute explicitly π D \\\\pi_{D} . If π C \\\\pi_{C} cannot be computed explicitly, our algorithm approximates it by numerical resolution of successive integral equations. Some numerical examples are studied to show the usefulness and proper functioning of our proposal.\",\"PeriodicalId\":46576,\"journal\":{\"name\":\"Monte Carlo Methods and Applications\",\"volume\":null,\"pages\":null},\"PeriodicalIF\":0.8000,\"publicationDate\":\"2023-03-30\",\"publicationTypes\":\"Journal Article\",\"fieldsOfStudy\":null,\"isOpenAccess\":false,\"openAccessPdf\":\"\",\"citationCount\":\"0\",\"resultStr\":null,\"platform\":\"Semanticscholar\",\"paperid\":null,\"PeriodicalName\":\"Monte Carlo Methods and Applications\",\"FirstCategoryId\":\"1085\",\"ListUrlMain\":\"https://doi.org/10.1515/mcma-2023-2003\",\"RegionNum\":0,\"RegionCategory\":null,\"ArticlePicture\":[],\"TitleCN\":null,\"AbstractTextCN\":null,\"PMCID\":null,\"EPubDate\":\"\",\"PubModel\":\"\",\"JCR\":\"Q3\",\"JCRName\":\"STATISTICS & PROBABILITY\",\"Score\":null,\"Total\":0}","platform":"Semanticscholar","paperid":null,"PeriodicalName":"Monte Carlo Methods and Applications","FirstCategoryId":"1085","ListUrlMain":"https://doi.org/10.1515/mcma-2023-2003","RegionNum":0,"RegionCategory":null,"ArticlePicture":[],"TitleCN":null,"AbstractTextCN":null,"PMCID":null,"EPubDate":"","PubModel":"","JCR":"Q3","JCRName":"STATISTICS & PROBABILITY","Score":null,"Total":0}
引用次数: 0
摘要
分划算法是显式计算有限马尔可夫链稳态概率的迭代过程𝑋。在本文中,我们提出将该算法应用于状态空间E:=C∪D E:=C \cup D由连续部分和有限部分𝐷组成,使得C∩D=∅C \cap D= \emptyset。在这种情况下,稳态概率𝑋分别是两条马尔可夫链的两个稳态概率π C \pi _C{和π }D \pi _D{的凸组合。得到的算法允许显式计算π D }\pi _D{。如果π C }\pi _C{不能显式计算,我们的算法通过连续积分方程的数值解析近似它。通过数值算例分析,说明了该方法的有效性和良好的功能。}
Computation of the steady-state probability of Markov chain evolving on a mixed state space
Abstract The partitioning algorithm is an iterative procedure that computes explicitly the steady-state probability of a finite Markov chain 𝑋. In this paper, we propose to adapt this algorithm to the case where the state space E := C ∪ D E:=C\cup D is composed of a continuous part 𝐶 and a finite part 𝐷 such that C ∩ D = ∅ C\cap D=\emptyset . In this case, the steady-state probability 𝜋 of 𝑋 is a convex combination of two steady-state probabilities π C \pi_{C} and π D \pi_{D} of two Markov chains on 𝐶 and 𝐷 respectively. The obtained algorithm allows to compute explicitly π D \pi_{D} . If π C \pi_{C} cannot be computed explicitly, our algorithm approximates it by numerical resolution of successive integral equations. Some numerical examples are studied to show the usefulness and proper functioning of our proposal.