A GIS-based serious game interface for therapy monitoring

Abstract

In this paper, we present a novel idea of a map-based therapy environment for people with Hemiplegia. The therapy environment is designed according to the suggestions of therapists, which consists of a spatial map browsing serious game augmented with our novel multi-sensory natural user interface (NUI). The NUI is based on 3D motion sensors that can recognize different hand and body gestures used for browsing a 3D or 2D map. The 3D motion sensors work in a non-invasive way; hence, they do not require any wearable body attachments and can be used at home without assistance from the therapists. The map-browsing environment provides an immersive experience to the disabled users, which helps in performing therapy in an interesting and entertaining manner. We have developed analytics for measuring certain quality of health improvement metrics from each type of spatial map browsing movements. The 3D motion sensors have been tested with Nokia, Google, ESRI, and a number of other maps that allow a subject to visualize and browse the 3D and 2D maps of the world. The map browsing session data shows the nature of big data; hence, the session data is stored in a cloud environment. Our developed serious game environment is web-based; thus anyone having the appropriate low cost sensor hardware can plug it in and start experiencing a natural way of hands free map browsing. We have deployed our framework in a hospital that treats Hemiplegic patients. Based on the feedback obtained, the developed platform shows a huge potential for use in hospitals that provide physiotherapy services as well as at patients' home as an assistive therapeutic service.