Software Testing, Verification and Reliability最新文献

In this issue, we are pleased to present two papers on test code evolution and mutation testing, respectively.

The first paper, “Towards automatically identifying the co-change of production and test code” by Yuan Huang, Zhicao Tang, Xiangping Chen and Xiaocong Zhou, presents a method named Jtup that uses machine learning to identify the cochange of production and test code. When a developer makes modifications to a class in production code, Jtup analyses the modified class and determines whether its corresponding test class needs to be modified as well. For machine learning, Jtup incorporates three types of features (code change features, code complexity features and code semantic features). The experimental results show the superior performance of Jtup in both within-project and multiclassification settings, surpassing multiple competing methods (Recommended by Wing Kwong Chan).

The second paper, “A new perspective on the competent programmer hypothesis through the reproduction of real faults with repeated mutations” by Zaheed Ahmed, Eike Schwass, Steffen Herbold, Fabian Trautsch and Jens Grabowski, presents a study of the competent programmer hypothesis based on the ability to reproduce faults through mutation operators. In contrast, previous work only considered how many tokens are changed by bugs or manually compared mutations with faults. The authors reframe the problem of transforming a correct into a buggy AST as a path search problem, where each step of a path is a mutation. The study results support the competent programmer hypothesis and also show that mutation operators are often not in line with the slight differences in correct code introduced by developers (Recommended by Marcio Delamaro).

We hope that these papers will inspire further research in related directions.

In this edition, we present two papers that offer significant contributions related to fuzz testing on one hand and vulnerability detection on the other hand, respectively, delving into directed greybox fuzzing (DGF) and tensor-based gated graph neural networks for automatic vulnerability detection in source code.

The first paper, ‘Greybox fuzzing, a scalable and practical approach for software testing’, by Pengfei Wang, Xu Zhou, Tai Yue, Peihong Lin, Yingying Liu and Kai Lu, proposes to go improve greybox fuzzing tools to uncover bugs, with directed greybox fuzzing (DGF). DFG emerges as a strategic alternative to undirected coverage-guided approaches, by allocating its resources purposefully, targeting specific zones like bug-prone areas. This makes DGF particularly effective for patch testing, bug reproduction and specialized bug detection scenarios. The paper conducts a comprehensive study, analysing 42 state-of-the-art fuzzers closely related to DGF. By categorizing DGF into location-directed and behaviour-directed types, the authors unveil its benefits, limitations and potential research avenues. This work not only provides a snapshot of the current state of DGF but also identifies gaps and proposes areas for future investigation.

The second paper, entitled ‘Tensor-based gated graph neural network for automatic vulnerability detection in source code’, is embracing the issue of the rapid expansion of smart devices that intensifies the demand for robust vulnerability detection in source code. Jia Yang, Ou Ruan and JiXin Zhang address this overall challenge by proposing a tensor-based gated graph neural network, named TensorGNN, for function-level vulnerability detection in source code. TensorGNN treats codes as graphs with node features by combining different code graph representations, leading to an accurate code embeddings. The TensorGNN model outperforms existing state-of-the-art works in terms of accuracy and F1 for vulnerability detection across various open-source code corpora. Notably, it achieves these results with significantly fewer training parameters and reduced training time. By introducing a novel perspective to vulnerability detection, this paper opens avenues for further exploration in the intersection of tensor technology and software security.

In conclusion, these two different papers contribute to complementary facets of software quality improvement. As STVR navigates the complexities of deploying safe and secure software, I wish you a pleasant reading that may inspire follow-up research in these two directions.

In this issue, we are pleased to present two papers on test case prioritization and mutation testing, respectively.

The first paper, ‘Semantic-aware two-phase test case prioritization for continuous integration’ by Yingling Li, Ziao Wang, Junjie Wang, Jie Chen, Rui Mou, and Guibing Li, presents the SatTCP framework to conduct precise prioritization with low time overhead, in order to improve the cost effectiveness of typical continuous integration (CI) testing with frequent code submissions. In SatTCP, coarse-grained filtering based on information retrieval (IR) techniques roughly sorts test cases and selects a certain number of tests for the subsequent prioritization; then fine-grained prioritization based on pretrained Siamese network conducts precise prioritization of initially ranked test sets. The evaluation results show that SatTCP outperforms all the baselines under comparison, and achieves the lowest test costs. (Recommended by Yves Le Traon).

The second paper, ‘Mutation testing optimisations using the Clang front-end’ by Sten Vercammen, Serge Demeyer, Markus Borg, Niklas Pettersson, and Görel Hedin, presents an investigation to which extent the Clang front-end and its state-of-the-art program analysis facilities allow to implement existing strategies for mutation optimization within the C language family. The authors develop a proof-of-concept tool used to collect detailed measurements for each mutation phase. The authors conduct evaluation of the proof-of-concept tool on four open-source C++ libraries and one industrial component. The evaluation results show that the ‘Generate Mutants’ and ‘Detect (Un)Reachable Mutants’ steps are for all practical purposes negligible; the ‘Compile Mutants’ step takes a significant amount of time and the compilation of the invalid and unreachable mutants is considerable; the ‘Execute Mutants’ step is the other dominant factor. (Recommended by Mike Papadakis).

We hope that these papers will inspire further research in related directions.