Controlled experiment on the comprehension of runtime phenomena using models created at design time

Abstract

Utilizing models for software construction is a well-studied research topic. Recent research investigates the integration of models into running systems to provide additional information about the system configuration at runtime. While this additional information enables innovative self-adaptive mechanisms, it is still an open research question if the information provided by models can also improve the analysis capabilities of human users where manual intervention is inevitable for investigating runtime phenomena. This paper contributes to fill this gap by conducting a controlled experiment where the correctness and completion time of tasks regarding runtime information are assessed. A control and experiment group had to analyze the output of a software system, and the experiment group additionally received traceability links between models and associated runtime records. The results show that improvements of the analysis can especially be observed where model elements emphasize relationships between system parts that are hardly recognizable in the implementation code.