A Teacher-Student Knowledge Distillation Framework for Enhanced Detection of Anomalous User Activity

Abstract

As information systems continuously produce high volumes of user event log data, efficient detection of anomalous activities indicative of insider threats becomes crucial. Typical supervised Machine Learning (ML) methods are often labor-intensive and suffer from the constraints of costly labeled data with unknown anomaly dependencies. Here we introduce a knowledge distillation ML framework, using multiple binary classifiers as teacher models and a multi-label model as the student. Leveraging the soft targets of teacher models, we demonstrate that the student model significantly improves performance.