An Experimental Evaluation of Web Mutation Operators

While modern web development technologies enhance the capabilities of web applications, they introduce challenges for testers. This paper introduces, evaluates, and refines web mutation operators that target interaction faults in web applications. An experimental study is conducted on 11 subject web applications using 15 web mutation operators. The effectiveness of 12 independently developed test sets are analyzed in terms of how well they kill web mutants. Web mutation adequate tests are compared with independently created test sets to evaluate the web mutation operators. Tests designed to satisfy the web mutation testing criterion provide 100% coverage while the tests designed to satisfy traditional testing criteria provide, on average, 47%coverage. The paper also analyzes which mutants and mutation operators the traditional tests had difficulty killing. We found that some types of mutants that are not particularly hard to kill were missed by all traditional tests. Additionally, web mutation testing produces very few equivalent mutants.