Evaluation of XAI Algorithms in IoT Traffic Anomaly Detection

Abstract

Anomaly detection in network traffic, both in general computer networks and specifically in Internet of Things (IoT) networks, plays a crucial role in ensuring computer network security. Over the years, numerous machine learning and deep learning-based anomaly detection tools have been proposed, exhibiting high accuracy in identifying anomalous behavior. However, a significant challenge arises with most machine learning and deep learning algorithms, as they are often considered black-box models that lack interpretability. Consequently, explaining the reasons behind certain network behaviors being labeled as anomalous becomes a difficult task. To overcome this issue, we evaluate the combination of anomaly detectors and eXplainable Artificial Intelligence (XAI) algorithms in IoT traffic anomaly detection. Our research results demonstrate that XAI algorithms can consistently identify the most impactful network features of security anomalies. More specifically, (1) SHAP algorithm is the most robust and reliable in the four tested XAI algorithms for four types of supervised/unsupervised anomaly detection models, independent of two datasets including different anomalies. (2) Image-based XAI algorithms are not suitable for explainability of network anomaly detection.