Optimal control of Markovian queueing systems

Dynamic optimization of queueing systems is treated by optimal control theory. This work is based on modelling the queueing problem as a time-varying continuous Markov chain. Necessary and sufficient conditions are given for a broad class of problems which include both scalar and Markovian dynamic programming control structures. Continuity of the switching function is used to characterize optimality near the end points of the horizon. Special properties of the model are exploited to ensure the absence of singular subarcs. Numerical results based on the use of a gradient algorithm report the effect of increasing the system capacity, a comparison of scalar versus Markovian dynamic programming controls, and an application to a multiprogrammed computer system.