Development of a video-based test for assessing decision-making proficiency in football referees

The aim of this study was to develop a valid and reliable video-based test to analyze the decision-making performance of Turkish football referees. Twenty-two divisional and fifteen local football referees affiliated to the Turkish Football Federation participated in the study. Using SuperLab 5 software and a Cedrus response pad, a test consisting of 20 video clips to be answered as yellow card, red card, and no card was designed. The referees watched the video clips in sequence and made a decision by pressing the button assigned for the decision of their choice. After each decision, the next video automatically appeared on the screen. The protocol was repeated at 10-day intervals for validity and reliability analyses. Content validity and construct validity were used for validity. Cohen’s kappa statistic and intraclass correlation coefficient were used for reliability. Expert referees confirmed content validity by assessing the degree to which the video clips reflected actual match conditions (CVR > 0.80). Difference analysis results showed that the test was able to discriminate the performance of the referees according to their level of performance, thus establishing construct validity ( p < 0.05). High Cohen kappa values indicated that the video clips were reliable ( K = 0.84). High ICC values indicated that the scoring scale was reliable and consistently assessed the decision-making performance of the referees (ICC = between 0.81 and 0.95; CV = between 0.74 and 4.49; MDC = between 0.02 and 1.45; SEM = between 0.01 and 0.52). In conclusion, the fact that the video-based decision-making test meets the validity and reliability criteria shows that the decision-making performance of the referees can be measured accurately and consistently.