The Supportive Effect of Traceability Links in Architecture-Level Software Understanding: Two Controlled Experiments

The advocates of architecture traceability approaches regularly cite advantages like easier understanding of architectural designs and support for software quality control and maintenance. However, the lack of published empirical data on the usefulness of architecture traceability is one of the reasons that prevents the wide adoption of traceability approaches in industrial settings. This paper reports on two controlled experiments performed with different participants to investigate whether the use of architecture traceability can significantly support architecture-level understanding activities. The replications with different participants allowed us to investigate whether the participants' experience plays a significant role in the understanding of software architectures with or without traceability information. In particular, we designed twelve typical questions aimed at gaining an architecture-level understanding of a representative subject system and measured how a control group (provided with no traceability information) and an experiment group (provided with traceability information) answered these questions in terms of the solutions' correctness and the participants' experience. Our findings show that the correctness of the answers of the participants in the experiment group is significantly higher than in the control group, whereas no significant differences with regard to the experience of the participants are observed.