Stochastic Modeling of Multi-state Disease Dynamics under Random Environments

This paper presents the application of Semi-Markov decision Process(SMDP) for a multi-state disease under random environments to determine the optimal treatment strategy. The subject/patient lives in varying random environments, imparting significant effects on performance/health status. While the environment evolves according to a Semi-Markov Process, in each environment state, the subject goes through several states of disease according to a Semi-Markov Process. In an environment 'k' when the patient state is 'i', one of the following two actions are available: continue the present treatment strategy (C) with a given cost rate hk(i) or initiate a rejuvenating treatment strategy (R) with a cost rate ck(i). In this complex model the optimal strategy is found out minimizing the expected discounted total cost. A special case of Markov environment is discussed indicating the feasibility of the computation of optimal policy. A numerical illustration is also provided to support the viability of the analysis and results. The model provides a useful and flexible representation of acute and chronic events and can be used to explore the economic impact of changes in therapy.