Reliability agreement in foul and penalty judgements between officials in the Swedish hockey league.

Abstract

Introduction: Officials are essential in terms of player safety and injury prevention, especially in contact team sports such as ice hockey, where numerous fast pace and high force contacts occur. If against the rules, these collisions can result in penalties. However, there is limited literature on the inter-rater reliability of the officials' decisions. Hence, the purpose was to investigate the theoretical reliability agreement between professional ice hockey officials in the Swedish Hockey League (SHL).

Method: Fifty video clips with different match situations were shown to 33 professional ice hockey officials in the SHL. Each situation was shown three times and the officials had 20 s between each video clip to answer which offence and penalty they would judge. The answers were anonymously collected using an online questionnaire. Fleiss' kappa was used to assess the reliability agreement between the referees, for each situation.

Results: The Fleiss' kappa values for all officials were 0.63 and 0.35 for offences and penalties, respectively. Referees and linesmen had similar kappa values for offences (0.64 vs. 0.64), as well as for penalties (0.38 vs. 0.35).

Conclusion: The results show that the suggested methodology can be used to identify situations where officials agree and disagree. In ice hockey, poor agreement regarding penalties can depend on the chosen offence as the rulebook limits the availability of penalties, based on the chosen offence. This can create issues, as there are situations where different offences are equally correct but will result in different penalties.