Principal component self-attention mechanism for melanoma hyperspectral image recognition

Abstract

Early detection of melanoma and prompt treatment are key approaches to reducing melanoma-related deaths. In order to improve the ability of early detection of melanoma, this paper introduces a set of hyperspectral images (HSIs) data captured by dermoscopy using hyperspectral technology, and based on this data, proposes a principal component self-attention mechanism (PCSAM) method for the classification of dysplastic nevus and melanoma. The proposed method uses principal component analysis technology to amplify the differences in spectral features of the lesions and extract some new features that are convenient for classification. In addition, under the action of the attention mechanism, the spectral features of melanoma are fully paid attention to, and the contextual spatial information between each HSI block can also be utilized. Finally, a comparison experiment is carried out using RGB images and HSIs. Experimental results demonstrate that the spectral features of melanoma can significantly improve the classification accuracy, and it also shows that the participation of hyperspectral technology can effectively improve the recognition accuracy of dysplastic nevus and melanoma, which reflects the advantages of HSI compared with the traditional image.