Identifying Parkinson’s Disease using Multimodal Approach and Deep Learning

Abstract

Parkinson’s disease is a progressive neurological disease resulting in motor impairments that decrease patients’ quality of life. It is a condition currently without a cure. This study proposes an approach to Parkinson’s disease detection using multimodal analysis, combining handwriting and voice data. We introduce the use of image transformer architectures to classify Parkinson’s disease patients from healthy subjects, as early diagnosis of Parkinson’s disease contributes to the management of motor symptoms. Data-efficient image transformer with self-supervised learning on DINO obtained an accuracy of above 90% on a combination of spiral and meander drawings from the NewHandPD dataset. In comparison, an audio spectrogram transformer obtained an accuracy of above 80% on the sustained vowel phonations of /a/ and /o/ from the PC-GITA corpus. This work considers using a multimodal approach in identifying Parkinson’s disease and the usability of transformer architectures in image and audio spectrogram classification tasks.