Test Optimization from Release Insights: An Analytical Hierarchy Approach

Abstract

Software Testing is an essential aspect to ensure software quality, reliability and consistent user experience. Digital applications such as mobile app usually follow rapid software delivery which consists of various releases. It typically uses insights from the development data such as defects, test logs for test execution optimization. Once the application is released and deployed, there is rich availability of untapped heterogeneous data which can also be effectively utilized for the next release test execution optimization. The data from the release includes direct customer feedback, application monitoring data such as user behavioral traces, device usages, release logs. In this position paper, we discuss about the various data sources and the multiple insights which can be derived from them. We also propose a framework which uses Analytical Hierarchy Process to prioritize the tests based on these insights available from the release data. The framework also recommends the prioritized and missed device configurations for next release test planning.