Win/loss States: An efficient model of success rates for simulation-based functions

Abstract

Monte-Carlo Tree Search uses simulation to play out games up to a final state that can be evaluated. It is well known that including knowledge to improve the plausibility of the simulation improves the strength of the program. Learning that knowledge, at least partially, online is a promising research area. This usually implies storing success rates as a number of wins and visits for a huge number of local conditions, possibly millions. Besides storage requirements, comparing proportions of competing patterns can only be done using sound statistical methods, since the number of visits can be anything from zero to huge numbers. There is strong motivation to find a binary representation of a proportion signifying improvement in both storage and speed. Simple ideas have difficulties since the method has to work around some problems such as saturation. Win/Loss States (WLS) are an original, ready to use, open source solution, for representing proportions by an integer state that have already been successfully implemented in computer go.