Multiscale feature fusion and enhancement in a transformer for the fine-grained visual classification of tree species

Abstract

Accurate and rapid fine-grained visual classification (FGVC) of tree species within the same family can provide technical support for tree surveys, research, and conservation. However, FGVC faces challenges such as large intraclass differences and small interclass differences. Recognizing tree species within the same family requires focusing on and correlating overall and multiorgan features of the trees while mitigating the influence of complex natural backgrounds, occlusion effects and other factors. To address these challenges, we propose multiscale feature fusion (MFF) and enhancement in transformers to improve recognition performance. The method consists of a Swin transformer backbone, an MFF module, a discriminative feature enhancement (DFE) module, and a texture feature enhancement (TFE) module. The MFF module aims to strike a balance between global and local feature extraction. The DFE module is employed to mitigate the impact of background noise, whereas the TFE module is used to enhance the feature extraction associated with complex textures and spatial patterns. We conducted experiments on a constructed dataset of tree species from the same family, achieving a top-1 accuracy of 90.3 % and a top-3 accuracy of 96.8 %. In addition, the method performed well on three popular FGVC datasets, namely, the Flavia, Oxford Flowers, and PlantCLEF 2015 datasets, with top-1 accuracies of 100 %, 99.2 %, and 81.4 %, respectively. The ablation experiments and module visualizations also yielded satisfactory results. Thus, this work provides a solution to enhance the FGVC task.