LMSFF: Lightweight multi-scale feature fusion network for image recognition under resource-constrained environments

Abstract

In many resource-constrained environments, recognition tasks often require efficient and fast execution. Currently, many methods designed for this field adopt a combination of convolutional operations and Vision Transformers (ViTs) to achieve comprehensive feature representation while maintaining efficient performance. However, these methods still have higher parameter counts or floating point operations (FLOPs), making it difficult to adapt more resource-constrained environments. Therefore, a lightweight Multi-Scale Feature Fusion Network (LMSFF) is proposed to address this issue. The proposed method mainly consists of three modules: lightweight local processing (LLP) modules, local–global fusion modules (LGFM), and lightweight information fusion (LIF) modules. The LLP modules, considering the issue of computational redundancy, propose a branch structure that effectively reduces parameter consumption while maintaining high performance. To capture more comprehensive contextual information, the LGFM fuses local and global features, thus enhancing the comprehensive representation of image features. The LIF extracts crucial features through pooling operations at different scales while preserving lightweight characteristics. Additionally, to enhance the model’s generalization, a new weighted loss function is introduced, which alleviates the long-tail distribution issue in real-world scenarios and improves recognition performance for rare categories. Experimental results demonstrate that LMSFF achieves better balance between recognition accuracy and resource consumption compared with other state-of-the-art lightweight hybrid models.