An adaptable multimodal crew assistance system for NATO generic vehicle architecture

Abstract

With the proliferation of new and improved sensors for military and civilian vehicles, the challenges for integrating the sensors, retrieving and processing data from them have become more complex. Typically, these sensors come with APIs as specified by the manufacturer and hence integrating newer sensors on the existing and newer platforms as well as interoperability with other platform sub-systems is highly taxing. Also, these sensors open the realm of possibilities for further automation of vehicles. Types and levels of automation are a big challenge as to how much automation is possible and should be allowed, and how to effectively use the sensors for automation and informing the vehicle crew. The NATO Generic Vehicle Architecture (NGVA) proposes an open architecture approach to land vehicle platform design and integration and standardises the interfaces and protocols for military vehicle systems integration. However, the NGVA does not specify any feedback mechanisms exploiting the available sensor data to support the vehicle crew. In this paper, we present a simple crew assistance system which integrates NGVA-compliant sensors and includes an adaptable rule-based interface for sensor data processing. This crew assistance system takes into consideration human factors such as human cognitive and physical workload and informs the crew members about ongoing critical events derived from available sensor data using visual, audio and haptic feedbacks in the form of alarms to increase their Situational Awareness by raising alarms on detection of specific events and making intelligent future predictions.