A Framework for Ensuring the Quality of a Big Data Service

During past several years, we have built an online big data service called CMA that includes a group of scientific modeling and analysis tools, machine learning algorithms and a large scale image database for biological cell classification and phenotyping study. Due to the complexity and “nontestable” of scientific software and machine learning algorithms, adequately verifying and validating big data services is a grand challenge. In this paper, we introduce a framework for ensuring the quality of big data services. The framework includes an iterative metamorphic testing technique for testing “non-testable” scientific software, and an experiment based approach with stratified 10-fold cross validation for validating machine learning algorithms. The effectiveness of the framework for ensuring the quality of big data services is demonstrated through verifying and validating the software and algorithms in CMA.