On Choice in Complex Environments

Abstract

A Decision Maker (DM) must choose at discrete moments from a finite set of actions that result in random rewards. The environment is complex in that she finds it impossible to describe the states and is thus prevented from application of standard Bayesian methods. This paper presents an axiomatic theory of choice in such environments. Our approach is to postulate that the DM has a preference relation defined directly over the set of actions which is updated over time in response to the observed rewards. Three simple axioms that highlight the independence of the given actions, the bounded rationality of the agent, and the principle of insufficient reason at margin are necessary and sufficient for the DM's preferences to admit a utility representation. The DM's behavior in this case will be akin to fictitious play. We then show that, if rewards are drawn by a stationary stochastic process, the observed behavior of such a DM almost surely cannot be distinguished from anyone who is fully cognizant of the environment.