Heuristic evaluation and end-user testing of a machine learning-based lower-limb exercise training system for management of knee pain in individuals aged 55 years or over

Objective

Using machine learning techniques, we have developed an interactive exercise training system to assist individuals aged 55 years or over with knee pain to perform lower-limb exercises to improve their knee health. The system has three features: video-based exercise demonstrations, real-time feedback on exercise movements, and tracking of exercise performance and progress. The current study aimed to evaluate the design of the computer prototype of the system, and determine its usability and end users’ intention to use it (i.e., acceptance of it).

Methods

Heuristic evaluation and end-user testing of the computer-based prototype system were conducted. Three human factors practitioners identified the design deficiencies, with reference to 64 design principles. In addition, 10 individuals with knee pain were recruited to use the prototype system to complete five tasks in the study laboratory. We recorded and examined the task success rate, number of requests for assistance, difficulties encountered during tasks, and perceptions of usability and acceptance.

Results

Four design deficiencies were identified, regarding recognition and recovery of errors, navigation, auditory perception, and help documentation. Most participants had difficulty in calibrating the camera and performing exercises. However, in general, the prototype system was perceived as usable and acceptable.

Conclusions

The use of heuristic evaluation and end-user testing revealed the capacity to systematically detect design deficiencies in interactive self-help systems, allowing for effective system adjustments. Moreover, our system shows potential for individuals managing knee pain, but conducting iterative usability testing is necessary to identify additional improvements. Furthermore, several design propositions have been submitted.