Privacy-preserving heterogeneous multi-modal sensor data fusion via federated learning for smart healthcare

Abstract

The widespread availability of medical Internet of Things devices and smart healthcare monitoring systems has unprecedentedly led to the emergence of the generation of heterogeneous sensor data throughout the different decentralized healthcare institutions. Although this data has a significant potential to enhance patient care, the handling of multi-modal sensor data, with the need to maintain the privacy of the patients and comply with the necessary regulations, proves to be very difficult using traditional ways of central processing. We propose PHMS-Fed, a novel privacy-preserving heterogeneous multi-modal sensor fusion framework based on federated learning for smart healthcare applications. Our framework enables healthcare institutions to train shared diagnostic models collaboratively without exchanging raw sensor data while effectively capturing complex interactions between different sensor modalities. In order to maintain the privacy of its use, PHMS-Fed, through adaptive tensor decomposition and secure parameter aggregation, automatically matches different combinations of sensor modalities across different institutions. The conducted extensive experiments on real-world healthcare datasets reveal the prominent effectiveness of the proposed framework, as PHMS-Fed has surpassed selected state-of-the-art methods by 25.6 % concerning privacy preservation and by 23.4 % in relation to the accuracy of the cross-institutional monitoring. As the results clearly show, the framework is extremely efficient in handling multiple sensor modalities while being able to deliver strong results in physiological monitoring (accuracy score: 0.9386 out of 1.0), privacy preservation (protection score: 0.9845 out of 1.0), and sensor fusion (fusion accuracy: 0.9591 out of 1.0) applications.