Automated testing of software-as-a-service configurations using a variability language

Abstract

The benefits offered by cloud technologies have compelled enterprises to adopt the Software-as-a-Service (SaaS) model for their enterprise software needs. A SaaS has to be configured or customized to suit the specific requirements of every enterprise that subscribes to it. IT service providers have to deal with the problem of testing many such configurations created for different enterprises. The software gets upgraded periodically and the configurations need to be tested on an ongoing basis to ensure business continuity. In order to run the testing organization efficiently, it is imperative that the test cycle is automated. Developing automated test scripts for a large number of configurations is a non-trivial task because differences across them may range from a few user interface changes to business process level changes. We propose an approach that combines the benefits of model driven engineering and variability modeling to address this issue. The approach comprises of the Enterprise Software Test Modeling Language to model the test cases. We use the Common Variability Language to model variability in the test cases and apply model transformations on a base model to generate a test model for each configuration. These models are used to generate automated test scripts for all the configurations. We describe the test modelling language and an experiment which shows that the approach can be used to automatically generate variations in automated test scripts.