ASGP-IDet: Temporal behaviour localisation of beef cattle in untrimmed surveillance videos

Abstract

An accurate analysis of beef cattle behaviour provides valuable information about their important characteristics such as health status and fertility. Recent studies have utilised computer vision technologies to recognise beef cattle behaviour in trimmed videos with a single behaviour. However, these methods ignore the fact that surveillance videos in real farm circumstances are usually untrimmed and contain multiple behaviour instances and background scenes, which limit their applicability. To address this issue, we propose a temporal behaviour localisation method using aggregate scalable-granularity perception instance detection (ASGP-IDet) to localise beef cattle behaviours in untrimmed videos. It provides semantic information, such as “ when does a specific behaviour start and end?” and “ duration of a specific behaviour”. To this end, a feature pyramid with ASGP blocks was designed to aggregate information across different temporal granularities. The trident head was then employed to achieve precise behaviour boundary predictions, and the classification head was used to predict the behaviour category of the instance. Finally, a novel centre–start–end instant offset loss (CSEIO Loss) is proposed for correct offsets at the start, end, and temporal centre of behaviours. Experiments on the newly collected Cattle Temporal Action dataset demonstrated that ASGP-IDet outperformed other state-of-the-art approaches. It achieved mAP scores of 93.93%, 93.74%, 93.22%, 92.29%, and 87.46% at tIoU thresholds [0.3:0.7:0.1], specifically, an average mAP of 92.13%, and an average processing time of 92.9 ms per video. These findings introduce an efficient method for localising the temporal behaviour of beef cattle in untrimmed farm surveillance videos and further support precision livestock farming.