Rationally inattentive Markov decision processes over a finite horizon

Abstract

The framework of Rationally Inattentive Markov Decision Processes (RIMDPs) is an extension of Partially Observable Markov Decision Processes (POMDP) to the case when the observation kernel that governs the information gathering process is also selected by the decision maker. At each time, an observation kernel is chosen subject to a constraint on the Shannon conditional mutual information between the history of states and the current observation given the history of past observations. This set-up naturally arises in the context of networked control systems, artificial intelligence, and economic decision-making by boundedly rational agents. We show that, under certain structural assumptions on the information pattern and on the optimal policy, Bellman's Principle of Optimality can be used to derive a general dynamic programming recursion for this problem that reduces to solving a sequence of conditional rate-distortion problems.