Objectivity of match analysis in football: Testing the level of agreement between coaches’ interpretations of video data

Abstract

Using video data is a widespread procedure in the preparation for an upcoming opponent across all levels of football, but the way coaches interpret this data and use it for player feedback is still not fully understood. Three studies were conducted to investigate the level of agreement between football coaches on tactical questions regarding the opponent when interpreting the same video data. In Study 1 (scouting feed; N = 15) and 2 (tactic view feed; N = 24), different video viewing angles of the same match were provided to coaches, followed by simple questions regarding the viewed team (e.g., team formation, most striking player in the opening play of the attacking team). Response analyses using descriptive statistics and Fleiss-Kappa statistics showed great diversity regardless of the angle of the feed. Study 3 was a replication study (scouting feed; N = 16) using the identical approach as before but used a different match to introduce greater variety of video stimuli. Across all studies there was a high degree of diversity in coach responses and little consensus on basic questions like the adopted formation or the most striking player in the opening play (Fleiss-Kappa coefficients between -.036 [poor agreement] and .236 [fair agreement]). The present research shows that it is problematic to treat information from video feeds as being objective when preparing for the next opponent, as different coaches derive different interpretations from the same data source. Implications for use of video data, and related contributions to coaching research are discussed.