Discovering anomalous behavior in large networked systems

Abstract

Tools for monitoring the performance and behavior of modern large networks produce an abundance of data, resulting in considerable interest in the ability to bring the most critical facets to the attention of human operators. While the coverage and sophistication of data being collected is expanding greatly to be comprehensive and detailed enough to solve hard problems, methods for analyzing this data tend to be either 1) too simplistic, resulting in too much information for users to process, many of which are false positives, or 2) too computationally intensive to keep up with the volume of data generated by large networks. We introduce a system that seeks a middle ground between these extremes using probability-based thresholding and temporal correlation of targeted, domain-specific network behavior metrics, resulting in fewer, higher-quality, more actionable events presented to users. In this paper we outline the problem area, present some of the mechanisms used, and then share two real examples of using anomaly detection to help large enterprises solve network problems.