New method for modeling digital twin behavior perception of cows: Cow daily behavior recognition based on multimodal data

Abstract

The cow digital shadow reflects the behavior, health condition, and productivity of cows, playing a crucial role in ensuring animal welfare, increasing individual productivity, and improving breeding efficiency. To fully utilize the existing multimodal data on farms and build a cow digital shadow with rich behavioral information, this study proposes a multimodal data fusion algorithm for recognizing cow behaviors such as drinking, feeding, lying, standing, and walking. This algorithm leverages the strengths of different data modalities, complementing each other, and enhances the performance of the cow behavior classification model. The algorithm integrates motion sensor and video data, collected by custom-made collars with inertial measurement units (IMUs) sensors placed at the top of the cow’s neck and cameras in the barn, using EfficientNet V2 S, BiLSTM, and Transformer networks. Experimental results demonstrate recognition accuracies of 98.80 %, precision of 97.15 %, and recall rates of 96.93 %, showing significant improvements over single-modal data behavior recognition algorithms. This method maximizes the utility of existing multimodal data to generate a cow digital shadow with detailed behavioral information, enhancing the modeling and simulation element of the cow digital twin architecture and laying the foundation for developing a comprehensive cow behavior data model.