A ranking method for large scale competitions based on sample grouping

In a large scale competition, the number of participants is too large to be completely reviewed by any evaluator, so dividing entire participants into several groups such that each of the evaluator reviews only a group serves as an alternative. However, in this case, an evaluator reviews and ranks only a portion of participants and the final result might be distorted. The focus of this study is on the methods of grouping participants and assignments of subgroups to evaluators. Under the assumption of not increasing the workload of evaluators, this study proposes a new method called sample ranking method, which according to simulation experiments yields better results than the commonly used ranking method.