Pattern Detection Model for Monitoring Distributed Systems

Abstract

The ever-increasing size, variety and complexity of distributed systems necessitate the development of highly automated and intelligent solutions for monitoring system parameters. In the context of Large Scale Distributed Systems, automatically detecting events and activity patterns will provide self-organization abilities and increase the dependability of these systems. We present in this paper a model for representing a wide variety of patterns in the parallel time series describing the distributed system parameters and states. Based on this model, we outline an application architecture for a system that employs advanced machine learning techniques for detecting and learning patterns in a distributed system with only minimal user input. The application is implemented as an add-on to the highly successful MonALISA monitoring framework for distributed systems. We test and validate the proposed model in real-time using the large amount of monitoring data provided by the MonALISA system. The novelty of this solution consists of the expressiveness of the model and the advanced automated data analysis for pattern learning and recognition in a long-time monitored system.