Benchmarking of Problems and Solvers: a Game-Theoretic Approach

Abstract In this note, we propose a game-theoretic approach for benchmarking computational problems and their solvers. The approach takes an assessment matrix as a payoff matrix for some zero-sum matrix game in which the first player chooses a problem and the second player chooses a solver. The solution in mixed strategies of this game is used to construct a notionally objective ranking of the problems and solvers under consideration. The proposed approach is illustrated in terms of an example to demonstrate its viability and its suitability for applications.