A relationship-aware mutual learning method for lightweight skin lesion classification

Abstract

In recent years, deep learning has made significant advancements in skin cancer diagnosis. However, most methods prioritize high prediction accuracy without considering the limitations of computational resources, making them impractical for wearable devices. In this case, knowledge distillation has emerged as an effective method, capable of significantly reducing a model's reliance on computational and storage resources. Nonetheless, previous research suffers from two limitations: 1) the student model can only passively receive knowledge from the teacher model, and 2) the teacher model does not effectively model sample relationships during training, potentially hindering the effective transfer of sample relationship-related knowledge during knowledge distillation. To address these issues, we employ two identical student models, each equipped with a sample relationship module. This design ensures that the student models can mutually learn while modeling sample relationships. We conducted extensive experiments on the ISIC 2019 dataset to validate the effectiveness of our method. The results demonstrate that our approach significantly improves the recognition of various types of skin diseases. Compared to state-of-the-art methods, our approach exhibits higher accuracy and better generalization capabilities.