Older Adults' Concurrent and Retrospective Think-Aloud Verbalizations for Identifying User Experience Problems of VR Games

While virtual reality (VR) games are beneficial for older adults to improve their physical functions and cognitive abilities, VR research often does not include older adults. Our review of the proceedings of major HCI conferences (i.e. ASSETS, CHI, CHI PLAY, CSCW and DIS) between 2016 and 2020 shows that only three out of 352 VR-related papers involved older adults. Consequently, older adults tend to encounter user experience (UX) problems with VR. One common way to identify UX problems is to conduct usability testing with think-aloud (TA) protocols. As VR games tend to be perceptually and physically demanding, older adults might need to allocate more resources to VR content and interaction and thus have fewer resources for thinking aloud. This raises the question of whether TA protocols are still a viable approach to detecting UX problems of VR games for older adult participants. To answer this question, we conducted usability testing with older adults who played two common types of VR games (i.e. the exergame and experience game) using concurrent and retrospective TA protocols (i.e. CTA and RTA), which are widely used in the industry. We analyzed participants' TA verbalizations and uncovered how different categories of verbalizations indicate UX problems. We further show how older adults perceived the effects of thinking aloud on their game experiences in two TA protocols and offer design implications.