Integrating Biofeedback and Artificial Intelligence into eXtended Reality Training Scenarios: A Systematic Literature Review

Abstract

BackgroundThe addition of biofeedback and artificial intelligence (AI) in simulation training and serious games has shown promising results in improving the effectiveness of training and can lead to increased engagement, motivation, and retention of information. This systematic literature review explores the integration of biofeedback and artificial intelligence into eXtended reality (XR) training scenarios and is the first review to provide a consolidated overview of applied biofeedback and AI technologies in this area.MethodThis review was conducted using keywords related to biofeedback, AI, XR, and training and included papers that: contained the use of biofeedback and AI in XR training scenarios; reported on at least one outcome related to training effectiveness; were published in English; were peer-reviewed; date from 1 January 2016 – 7 February 2022.ResultsThe results indicate that many studies collect two or more biosignals using a single biosensing device. This is particularly relevant in applied settings, where ease of use and minimal interference in training/education activities is desired. Also, that light, portable devices such as wrist bands, wireless straps, or headbands are preferred. Additionally, eye tracking, electrodermal activity (EDA), and photoplethysmograms (PPG) present as particularly useful biomarkers of stress and/or cognitive load in XR training contexts. A wide variety of machine learning (ML) approaches were used to support biofeedback systems in XR environments. However, a limited number of studies employed real-time analysis of biosignals (just 1% of studies) which indicates current challenges in implementing such systems.ConclusionThe majority of papers meeting the selection criteria were from the fields of education and healthcare. Further research in other domains, such as defense and general industry, is needed to gain a comprehensive understanding of the potential for biofeedback and AI integration in XR training scenarios used in these domains.