Statistical selection of the best system

Abstract

This tutorial discusses some statistical procedures for selecting the best of a number of competing systems. The term "best" may refer to that simulated system having, say, the largest expected value or the greatest likelihood of yielding a large observation. We describe various procedures for finding the best, some of which assume that the underlying observations arise from competing normal distributions, and some of which are essentially nonparametric in nature. In each case, we comment on how to apply the above procedures for use in simulations.