AR-Coach: Using Augmented Reality (AR) for Real-Time Clinical Guidance During Medical Emergencies on Deep Space Exploration Missions

Space travel imposes significant risks to crew health due to physiological adaptations, exposure to physical and psychological stressors, and limited capabilities to provide medical care. When medical emergencies occur, appropriate use of diagnostic and procedural guidance tools are crucial countermeasures against the risks of injury and mission failure. Point-of-care ultrasound (POCUS) is the only portable imaging modality available during exploration missions that can provide critical and dynamic medical information. Developing competency in POCUS is time-consuming and it is usually achieved through years of medical residency or clinical fellowship training programs. Due to the amount of astronaut training currently required, it is not feasible to also provide them with in-depth POCUS training. Current cognitive aids for POCUS-based procedures are either paper-based or static electronic checklists, which can be cumbersome to use, non-intuitive, and sometimes distracting; applying their written guidance to real actions can be difficult. To overcome these limitations, we developed a proof of concept of an augmented reality (AR) Coach (AR-Coach) as an Augmented Clinical Tool (ACT): a hands-free virtual coach system that guides the crew in real-time on how to perform POCUS during medical emergencies in space. To better understand the context and design requirements for the proposed ACT, we applied a human-centered design approach as part of our wider space medicine research program. We convened a multidisciplinary expert panel (n=46), including astronauts, flight surgeons, clinicians, XR and AI experts, to identify essential capability requirements. Task analysis with five experts, including clinicians, human factors researchers, and an XR developer, was used to create a process model of a POCUS-guided procedure to diagnose a potentially life-threatening condition (i.e., pneumothorax) that could occur during space travel. An iterative design and prototyping process was conducted. Informed by the expert panel and task analysis, we created a proof of concept of the AR-Coach which includes holographic panels that guide the crew in confirming the diagnosis of pneumothorax using POCUS. The results of this study can be applied in advancing space technologies that support astronauts in managing medical events during space exploration missions, optimizing performance, and improving crew safety.