A novel approach for application classification with encrypted traffic using BERT and packet headers

Recent years have seen substantial advancements in Internet technology along with environmental changes, which have led to the emergence of various security issues. There is also a trend of explosive growth in applications that encrypt network traffic for various types of services. Therefore, the classification of applications within encrypted traffic represents an important research issue for both secure network management and efficient bandwidth management. In such encrypted traffic, the payload itself is encrypted, and it is no longer viable to classify applications based on signatures extracted from plaintext. Most applications in public datasets for encrypted traffic classification are collected with the same IP address and port number, which makes the 5-tuple information a strong identifier. However, this 5-tuple contains many characteristics related to both the traffic collection environment and user-specific traits, rather than intrinsic features of the applications themselves. Therefore, when addressing the problem of encrypted traffic application classification, it is advisable to utilize header information excluding the 5-tuple and payload. Therefore, this paper proposes a novel service type and application classification system based on the Bidirectional Encoding Representation Transformer (BERT), which utilizes packet header information from encrypted traffic. The proposed system ensures the accuracy and generalization performance of the classification model by using only the header information from traffic packets, excluding the 5-tuple and payload. Further, to preserve the characteristics and semantic meaning of an encrypted traffic packet, sentences embedded with 2-byte tokens were used as input for BERT. The proposed system was designed to exclude labeling information from all sentences during the pre-training phase before proceeding with training. Fine-tuning was then conducted to align the system with the objectives of the service type and application classification. This experiment utilized the publicly available ISCX VPN-nonVPN dataset, and the proposed model achieved remarkable accuracy in the key performance measure, i.e., F1-scores, with values of 99.24 % in service type classification and 98.74 % in application classification. This capability can be used in maintaining the confidentiality of encrypted traffic, network security monitoring, Quality of Service (QoS), and traffic management in complex IT environments.