A butterfly stroke swimming recording and performance analysis system based on computer vision and machine learning

Abstract

Motion capture technology, which uses wearable devices and cameras, is critical for tracking and recording athletes’ movements. However, most wearable devices are unsuitable for water use, and manually analyzing swimming data from video footage is labor-intensive. Consequently, this study aims to develop an intelligent swimming performance recording system that employs cameras and accelerometers to assess the butterfly stroke. A swimmer recognition model is developed by applying transfer learning to 120,031 labeled images, utilizing YOLOv4 as the pre-trained model. Calculating key metrics using four generated bounding box coordinates such as 15 m/25 m split times, stroke count, and knee angles during underwater dolphin kicks. Subsequently, accelerometer data is synchronized based on timing and stroke count during the butterfly stroke. 85 generated videos of five swimmers performing 25-meter butterfly strokes were analyzed to validate the system. The swimmer recognition model achieved an intersection over union (IoU) of 81.51 %, while the mean absolute error (MAE) for split time measurements was 0.36 s, and the F1-score for stroke identification was 97.85 %. In conclusion, this system offers a valuable tool for providing users with prompt, comprehensive, quantified, and visual feedback following each practice session.