Using Evolutionary Testing to Find Test Scenarios for Hard to Reproduce Faults

Abstract

In industrial practice, developers are often unable to reproduce errors that are encountered by end-users or testers. Evidently, reproducibility is important for investigating the root cause of the error, since without knowing what causes the error, a developer cannot repair the software. This paper reports on the successful application of evolutionary testing by Rila Solution EAD to solve two real reproducibility problems they encountered. Rila's software application, the ChatPC, suffered from memory and data corruption faults, that were reported by users in the field but could not be reproduced in-house after considerable effort. This paper presents two case studies that show how evolutionary testing resulted in finding execution scenarios that could reliably reproduce the mentioned faults observed in the application.