An Empirical Study of Software Testing Quality based on Natural Experiments

Abstract

Software testing is an indispensable part of the software life cycle, and the quality of software testing largely affects the quality of software delivered to users. However, in the current stage of software testing quality research, the focus on testing quality influencing factors is still limited to theoretical analysis and model application, and there is a lack of empirical research based on real data and samples. To address this problem, this paper proposes the idea of using natural experiments to conduct empirical research in the field of software testing quality for the first time, and analyzes the feasibility of the research proposal through literature research. In this paper, the empirical study was conducted using real data provided by enterprises, through the research of cooperative enterprise project data and event records, selected the “CNAS expansion of the on-site review and audit training” as an exogenous event, team capacity as the explanatory variable to build a natural experimental model of test quality. After completing the empirical model, we analyze the results, which show that the exogenous events have a significant disposition effect on test quality, and the empirical results pass the four commonly used robustness tests, indicating that the experimental results have a high 95% confidence level. In addition, this paper also analyzes the control variables in the empirical model and finds that test team size can have an impact on test quality by affecting test diversity, which provides ideas for subsequent research. Finally, based on the experimental ideas and empirical results of this study, this paper summarizes the methodological paradigm of applying the natural experiment method to empirically study and analyze the factors affecting test quality, which provides an important reference example for future empirical studies by introducing the natural experiment method in the study of software quality and test quality, and greatly expands the research horizon of related fields.