A Comparison of Constraint-Based and Sequence-Based Generation of Complex Input Data Structures

Abstract

Generation of complex input data structures is one of the challenging tasks in testing. Manual generation of such structures is tedious and error-prone. Automated generation approaches include those based on constraints, which generate structures at the concrete representation level, and those based on sequences of operations, which generate structures at the abstract representation level by inserting or removing elements to or from the structure. In this paper, we compare these two approaches for five complex data structures used in previous research studies. Our experiments show several interesting results. First, constraint-based generation can generate more structures than sequence-based generation. Second, the extra structures can lead to false alarms in testing. Third, some concrete representations of structures cannot be generated only with sequences of insert operations. Fourth, slightly different implementations of the same data structure can behave differently in testing.