Using Web Server Logs to Identify and Comprehend Anomalous User Activity

Abstract

This research paper presents a study for identifying user anomalies in large datasets of web server requests. Using a cybersecurity company's network of web servers as a case study, we propose a technique for analyzing user activity in NGINX logs. The proposed method does not require a labeled dataset and is capable of efficiently identifying different user anomalies in large datasets with millions of daily requests. The results of the analysis provided a deeper understanding of user behavior when seeking updates through web requests and aided in interpreting the findings. Clustering the anomalies helped to produce typical clusters and further supported the interpretation of the results. This work provides valuable insights into user behavior in web server networks and highlights the importance of efficient anomaly detection techniques in large datasets. The findings have potential real-world applications in the field of cybersecurity, particularly in providing network security analysts with an automated and more objective approach to threat analysis. This study showcases the importance of automated methods for analyzing user activity in web server networks and provides a more objective and efficient approach to detecting user anomalies in large datasets. This approach contributes to the development of more effective and precise cybersecurity systems, ultimately improving the protection of network infrastructures from malicious attacks.