DeepKernel: 2D-kernels clustering based mutant reduction for cost-effective deep learning model testing

Abstract

Mutation testing is a practical approach for evaluating the quality of deep learning (DL) testing datasets. However, the enormous mutants during testing lead to significant testing overhead. Feature clustering is a conventional method that reduces the number of mutants while preserving the mutants’ distribution diversity. This distribution diversity is considered crucial for maintaining the effectiveness of testing assessment ability. DL model relies on convolutional kernels to extract data features and construct logic. Thus, using kernels to measure the differences among DL mutants is a feasible approach. This paper proposes DeepKernel, a convolutional kernel features clustering based reduction method. Specifically, it considers 2D-Kernel sparsity and 2D-Kernel entropy as kernel features. The features are clustered to construct a subset with equivalent testing assessment capability to the original set. Empirical studies on four classical DL models demonstrate that: (1) there is a significant correlation between the distribution diversity of the mutants and their testing assessment ability, as indicated by a Spearman Correlation Coefficient of 0.9689. (2) the reduced set maintains a similar distribution diversity and testing effectiveness as the original set. (3) when preserving the effectiveness of the mutation testing, our method reduces 63.47% of mutants and outperforms random selection.

Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.