How to test your concurrent software: an approach for the selection of testing techniques

Abstract

High-Performance Computing (HPC) applications consist of concurrent programs with multi-process and/or multithreaded models with varying degrees of parallelism. Although their design patterns, models, and principles are similar to those of sequential ones, their non-deterministic behavior makes the testing activity more complex. In an attempt to solve such complexity, several techniques for concurrent software testing have been developed over the past years. However, the transference of knowledge between academy and industry remains a challenge, mainly due to the lack of a solid base of evidence with information that assists the decision-making process. This paper proposes the construction of a body of evidence for the concurrent programming field that supports the selection of an adequate testing technique for a software project. We propose a characterization schema which assists the decision-making support and is based on relevant information from the technical literature regarding available techniques, attributes, and concepts of concurrent programming that affect the testing process. The schema classified 109 studies that compose the preliminary body of evidence. A survey was conducted with specialists for the validation of the schema, regarding adequacy and relevance of the attributes defined. The results indicate the schema is effective and can support testing teams for concurrent applications.