Shifted Hexpo activation function: An improved vanishing gradient mitigation activation function for disease classification

Abstract

Activation functions (AFs) in deep learning significantly impacts model performance. In this study, we proposed Shifted Hexpo (SHexpo), an improved variant of the Hexpo AF, designed to address limitations such as vanishing gradients and parameter sensitivity. SHexpo introduces a shifting parameter, enhancing its adaptability and performance across diverse data distributions. Using ResNet 101, DenseNet 169, 5 and 10-layer lightweight Convolutional Neural Network (CNN) trained on the SIPaKMeD dataset for cervical cancer classification, we compared SHexpo against Hexpo, ReLU, Swish, Mish, GELU and PReLU under four pre-processing techniques: zero-mean centering, normalization, their combination and ImageNet weights. Our results demonstrate that SHexpo achieves higher classification accuracy and better gradient stability than Hexpo while performing competitively with state-of-the-art AFs. Our findings indicate that SHexpo can be effectively integrated into both lightweight and deep architectures. Additionally, Grad-CAM visualizations highlight SHexpo’s capability to enhance interpretability by localizing the most relevant image regions contributing to model predictions. These results demonstrate SHexpo’s potentials for medical image analysis in low-resource settings.