Test Code Reuse from OSS: Current and Future Challenges

Abstract

We are told of the significant benefits of automated approaches to testing over manual approaches. However, it is unclear what automated testing practices exist, and how efficient or widespread such practices are within open source software. Although some organizations rigorously apply automated testing to their software, this rich pool of test code is not utilized to serve existing source code with poor or no test suites. To investigate how automated testing is performed in practice, we attempted a thorough, large-scale analysis of open source repositories. Alongside this analysis, we propose a novel approach to reuse such existing tests within projects that lack test code, hence leveraging the quality of such projects with minimal developer intervention. While such an analysis seems to be a straightforward task, we report on various practical challenges that hindered applying our proposed approach for tests' reuse. We present the challenges we have addressed so far, and those we expect to appear in the near future, in applying our approach for test reuse with open source projects. We outline potential solutions to the projected future challenges.