M.R. Hunter: Hunting for Metamorphic Relations by Puzzle Solving

Abstract

Metamorphic testing (MT) is getting increasingly popular by exhibiting test effectiveness for a wide range of subjects, from compilers to machine learning programs. However, the central part of MT, i.e., the derivation of useful metamorphic relations (MRs), still falls behind MT's rapid applications. In this paper, we propose M.R. Hunter, an interactive online game for attracting users, especially those who are not familiar with, or even reluctant to, learning intrinsic complexities behind MT, to participate into the MR derivation process in a puzzle-solving way. The game design carefully considers how to guide users to participate actively, how to present conjectured MRs intuitively, and how to validate MRs effectively. So far, we have built and deployed a preliminary version of the game, and received active feedbacks, suggesting both promising results and useful advices for future improvement.