Assessment of Driver's Stress State Using Smart T-Shirt Textile Electrodes and Multimodal Cross-Attention Networks

Abstract

Textile sensors enable noninvasive health monitoring, crucial for ensuring road safety by conducting mental well-being checks for drivers. Assessing driver stress with multimodal data from textile electrodes requires effectively integrating and interpreting diverse physiological signals and kinematic data. In this study, we evaluated textile electrodes and cross-attention mechanisms for assessing driver stress using multimodal data. Electrocardiography and respiration data were collected from 15 healthy volunteers wearing smart shirts in two driving scenarios. Signals were sampled at 256 and 128 Hz, respectively, with vehicle data also recorded. Segmented physiological and vehicle data enter separate networks, 1-D convolutional layers for signals and fully connected layers for vehicle data. Cross-attention fuses physiological data; these features are combined with vehicle data for stress classification using sigmoid. The proposed approach is able to classify driver stress states using multimodal data, achieving an average accuracy of 79 ${\%}$ and an average F-score of 75 ${\%}$ . The integration of a cross-attention mechanism facilitates the capture of intermodality information.