In Search of Distance Functions That Improve Autoencoder Performance for Intrusion Detection

Expectations of detection systems have risen with the increase in cyber-attacks. In order to detect the latest and future attacks, systems capable of detecting unknown attacks are needed. Among the various approaches offered by machine learning models, anomaly detection methods can address this need. It is possible to use the autoencoder to detect anomalies and therefore attacks. An autoencoder trained on data from normal use is able to detect attacks, unknown to the model. The attack detection is possible by observing the reconstruction error, which is the distance between the input and the reconstructed input resulting from the model. We considered different distance functions to improve the separation between attacks and normal events, and thus, to improve the performance of the autoencoder. We propose to use the cosine function of the angle formed between the actual input vector and the reconstructed input vector, as a distance function to address the problem of overlapping between normal events and attacks. In addition, we used Tree-structured Parzen Estimator algorithm for the optimization of the hyperparameters of the model. We ran our method on the NSL-KDD dataset and compared the obtained results to those of other methods that exist in the literature.