TMVF: Trusted Multi-View Fish Behavior Recognition with correlative feature and adaptive evidence fusion

Abstract

Utilizing multi-view learning to analyze fish behavior is crucial for fish disease early warning and developing intelligent feeding strategies. Trusted multi-view classification based on Dempster–Shafer Theory (DST) can effectively resolve view conflicts and significantly improve accuracy. However, these DST-based methods often assume that view source domain data are “independent”, and ignore the associations between different views, this can lead to inaccurate fusion and decision errors. To address this limitation, this paper proposes a Trusted Multi-View Fish (TMVF) Behavior Recognition Model that leverages adaptive fusion of associative feature evidence. TMVF employs a Multi-Source Composite Backbone (MSCB) at the feature level to integrate learning across different visual feature dimensions, providing non-independent feature vectors for deeper associative distribution learning. Additionally, a Trusted Association Multi-view (TAMV) Feature Fusion Module is introduced at the vector evidence level. TAMV utilizes a cross-association fusion method to capture the deeper associations between feature vectors rather than treating them as independent sources. It also employs a Dirichlet distribution for more reliable predictions, addressing conflicts between views. To validate TMVF’s performance, a real-world Multi-view Fish Behavior Recognition Dataset (MFBR) with top, underwater, and depth color views was constructed. Experimental results demonstrated TAMV’s superior performance on both the SynDD2 and MFBR datasets. Notably, TMVF achieved an accuracy of 98.48% on SynDD2, surpassing the Frame-flexible network (FFN) by 9.94%. On the MFBR dataset, TMVF achieved an accuracy of 96.56% and an F1-macro score of 94.31%, outperforming I3d+resnet50 by 10.62% and 50.4%, and the FFN by 4.5% and 30.58%, respectively. This demonstrates the effectiveness of TMVF in multi view tasks such as human and animal behavior recognition. The code will be publicly available on GitHub (https://github.com/crazysboy/TMVF).