Abstract: Autonomic Modeling of Data-Driven Application Behavior

Abstract

Computational behavior of large-scale data driven applications is a complex function of their input, configuration settings, and underlying system architecture. Difficulty in predicting the behavior of these applications makes it challenging to optimize their performance and schedule them onto compute resources. However, manually diagnosing performance problems and reconfiguring resource settings to improve application performance is infeasible and inefficient. We thus need autonomic optimization techniques that observe the application, learn from the observations, and subsequently successfully predict application behavior across different systems and load scenarios. This work presents a modular modeling approach for complex data-driven applications using statistical techniques. These techniques capture important characteristics of input data, consequent dynamic application behavior and system properties to predict application behavior with minimum human intervention. The work demonstrates how to adaptively structure and configure the models based on the observed complexity of application behavior in different input and execution scenarios.