Continuous-time restless bandit and dynamic scheduling for make-to-stock production

We study the "Whittle relaxation" version of the continuous time, discrete, and continuous state space Restless Bandit problem under the discounted cost criterion. Explicit expressions of Whittle's priority indexes, which generalize the Gittins indexes, are derived. This formalism is then used in the context of flexible make-to-stock production to construct dynamic scheduling rules. These analytical results are finally compared with the optimal numerically derived policy, obtained for a server delivering two product types. It is observed that the Whittle relaxation version of the Restless Bandit model nearly yields optimal dynamic scheduling rules.