Beacon: Automated Test Generation for Stack-Trace Reproduction using Genetic Algorithms

Software crashes are a problem all developers face eventually. Manually reproducing crashes can be very expensive and require a lot of effort. Recent studies have proposed techniques to automatically generate tests to detect and reproduce errors. But even if this topic has been widely studied, there has been little to no progress done for dynamically typed languages. This becomes important because current approaches take advantage of the type information inherent to statically typed languages to generate the sequence of instructions needed to reproduce a crash, thus making it unclear to judge if type information is necessary to reproduce errors. The lack of explicit type declarations in dynamic languages difficults the task of generating the instructions to replicate an error because the type checking can only be done during runtime, making algorithms less knowledgeable about the program and, therefore, making it more difficult to use search-based approaches because the algorithms have less information to work with. This paper presents a Genetic Algorithm approach to produce crash reproductions on Python based only on the information contained in the error's stack-trace. An empirical study analysing three different experiments were evaluated giving mostly positive results, achieving a high precision while reproducing the desired crashes (over 70%). The study shows that the presented approach is independent of the kind of typing of the language, and provides a solid base to further develop the topic.