Anomaly Detection in Intrusion Detection System using Amazon SageMaker

Abstract

Applying artificial intelligence and machine learning to analyzing network traffic has the potential to be transformative in protecting organizations from cyber threats. Intrusion detection systems (IDS) are historically rule-based; however, they could be improved. Applying machine learning in the form of Anomaly Detection could be the next step in preventing cyber threats from causing malicious activity on the network. Two algorithms that are implemented in anomaly detection through the use of Amazon SageMaker are Random Cut Forest (RCF) and XGBoost. The data for this project are the training and testing data set provided by the UNSW-15 data set. The models are created using the Jupiter Notebook on the Amazon SageMaker Studio Lab platform. The models were tested using the metrics of accuracy, precision, recall, and F1 score. The best-performing model was the XGBoost model, with an accuracy of 61.83%. The recall for this model was 96.49%, and the f1 score was 73.24%.