Using Model Scoping with Expected Model Elements to Support Software Model Inspections: Results of a Controlled Experiment

Context: Software inspection represents an effective way to identify defects in early phase software artifacts, such as models. Unfortunately, large models and associated reference documents cannot be thoroughly inspected in one inspection session of typically up to two hours. Considerably longer sessions have shown a much lower defect detection efficiency due to cognitive fatigue. Goal: The goal of this paper is to propose and evaluate a Model Scoping approach to allow inspecting specific parts of interest in large models. Method: First, we designed the approach, which involves identifying Expected Model Elements (EMEs) in selected parts of the reference document and then using these EMEs to scope the model (i.e., remove unrelated parts). These EMEs can also be used to support inspectors during defect detection. We conducted a controlled experiment using industrial artifacts. Subjects were asked to conduct UML class diagram inspections based on selected parts of functional specifications. In the experimental treatment, Model Scoping was applied and inspectors were provided with the scoped model and the EMEs. The control group used the original model directly, without EMEs. We measured the inspectors’ defect detection effectiveness and efficiency and collected qualitative data on the perceived complexity. Results: Applying Model Scoping prior to the inspection significantly increased the inspector defect detection effectiveness and efficiency, with large effect sizes. Qualitative data allowed observing a perception of reduced complexity during the inspection. Conclusion: Being able to effectively and efficiently inspect large models against selected parts of reference documents is a practical need, in particular in the context of incremental and agile process models. The experiment showed promising results for supporting such inspections using the proposed Model Scoping approach.