Deep Learning for Semantic Segmentation of Football Match Image

Abstract

As one of the most popular sports, football has been a subject to growth and advancements in technology. The combination of football and artificial intelligence is expected to be used for intelligent football analysis. Image semantic segmentation is an important basis for image analysis and understanding. This paper proposes a deep learning-based image segmentation model for pixel-level classification of the video recordings frames of football matches. Every pixel of football video frame is classified into one of the 10 classes, e.g., players, ball, goal bar and several background scenes. In this paper, we first test a variety of CNN architectures and pre-trained models and select the MobileNet-UNet architecture as our baseline. We note the severe unbalanced data distribution in football scene segmentation. To solve this problem, the weighted multi-class cross-entropy loss is adopted in training of MobileNet-UNet to redistribute the weights of classification loss, focusing on smaller foreground object classes and improving segmentation accuracy. We also propose to use image transformations and a random mixture sampling technique for training data augmentation to reduce model overfitting. The model is trained and validated in the well-annotated Football Semantic Segmentation Open Dataset. The proposed best model achieves 0.96 frequency weighted IoU and 0.90 mean IoU segmentation accuracy on validation set.