A comparative analysis of usability evaluation methods of academic mobile application: are four methods better?

Abstract

Usability evaluation is an important element which has used to add on insights related to the usability problem of an application. This study aims to identify application's usability problems and to compare the effectiveness of four evaluation methods which has used on an academic portal mobile application: usability testing, interviews, surveys, and heuristic evaluation. The data has collected from users who are in the student category and expert evaluator. The number detail of respondents used are usability testing (N = 10), interviews (N = 10), surveys (N = 110), and heuristic evaluation (N = 3). The four methods identified a total of 44 usability problems: 45% using heuristic evaluation, 24% using surveys, 17% using interviews and 14% using usability testing, resulting into a few similar findings. Then, The problems are categorized using Usability Taxonomy Problem (UPT) which has divided into 5 categories with details of 17 categories of visualness, 6 categories of language, 3 categories of manipulation, 11 categories of task-mapping and 7 others including the category of task-facilitation. The results of this study are capable to prove that the four methods are complementary, each method provides a unique insight to improve the usability of the application user interface. Both researchers recommend using a multi-method approach when evaluating the usability of an application due to it could provide a more comprehensive representation of usability issues.