SCAREX: A framework for scalable, reliable, and extendable cluster computing

Abstract

Cluster computing has been widely used in various fields from academics to industry because of its technical advantages. Recently, cloud computing become more attractive, as cluster computing is one of the core basements of cloud computing. However, the development of cluster-based system is hard without relative knowledge such as networking, workload distribution, fault tolerant, etc. Thus, various implementations for cluster computing have been introduced to help the development of cluster-based system. However, these implementations also have the limitation for developing cluster-based system because of high performance in specified application fields. In this paper, we present the SCAREX framework which supports scalability, reliability, extendibility, and easy for implementing cluster-based system. This framework employs a fault-tolerant mechanism for the failure of workload processing, and facilitates dynamic module distribution and loading for the work module which works on the worker node. The SCAREX also provides a simple programming interface for easy development. Since the SCAREX has flexible architecture, the developer can implement various cluster architecture and extend cluster architecture by combining the SCAREX-based cluster with other packages of the SCAREX or other frameworks.