A multi-view automatic table tennis umpiring framework

Abstract

This paper presents a low-cost, portable, multi-view table tennis umpiring framework, as a viable alternative to the current expensive systems which are almost exclusively restricted to elite professional sports. Table tennis has been selected as the sport to evaluate this framework primarily because it comprises many different complex match elements, including the service, return and rally elements, which are governed by a strict set of regulations which need to be umpired. The aim is to develop novel methods to analyse and judge the legality of such key match facets, with ball detection and tracking in video frames being integral to reliably and accurately determining the ball’s position and flight path during rallies. While a low-cost option is attractive and offers several benefits, it is a technically challenging problem due to the small number and generally low-resolution cameras that are used. A novel multi-view camera setup and multi-agent system (MAS) framework is presented, which comprises computationally lightweight agents which detect and track the table tennis ball, create a 3D representation of the flight path of the ball, predict the ball’s trajectory, and identify and analyse key facets in a table tennis rally. The MAS correctly detects all state transitions in seven test table tennis sequences with minimal latency and while the processing rate of a standard computer may be unable to analyse long rallies in real time, the potential of running the MAS on a parallel architecture is a propitious alternative. The MAS is also scalable, enabling additional camera pairs to be deployed to achieve enhanced accuracy and coverage. The framework affords the potential to reform the way matches are umpired, especially for amateur players, providing an economic and objective manner of dispute resolution, while the multi-view facility is extendible to other relevant ball-based sports. The ball flight path analysis mechanism can be exploited as a valuable training tool for skill development.