Experiment on UI evaluation using automated test

Learnability is one important aspect of user interaction that measures how long a user needs to familiarize themselves with the software. The evaluation method using expert analysis or user questionnaire cannot fully capture the learnability aspect of a software. Automated testing can record the user performance data and provide an objective evaluation of learnability. However, embedding recording code to conduct automated test can be expensive. This work proposes a novel method of automatic testing to evaluate the learnability of an existing software. By using Figma and Maze apps, a replica of evaluated software is made and injected with users’ performance recording module with much ease. The result of the experiment shows that learnability data can be acquired objectively. In the experiment, the user of evaluated software requires an average learning rate of 3 iterations. While the average completion time is around 2.37 seconds per action for trained respondents and 1.86 seconds for untrained respondents.