Dual stream transformer for medication state classification in Parkinson’s disease patients using facial videos

Abstract

Hypomimia is a prominent, levodopa-responsive symptom in Parkinson’s disease (PD). In our study, we aimed to distinguish ON and OFF dopaminergic medication state in a cohort of PD patients, analyzing their facial videos with a unique, interpretable Dual Stream Transformer model. Our approach integrated two streams of data: facial frame features and optical flow, processed through a transformer-based architecture. Various configurations of embedding dimensions, dense layer sizes, and attention heads were examined to enhance model performance. The final model, trained on 183 PD patients, attained an accuracy of 86% in differentiating between ON- and OFF-medication state. Moreover, uniform classification performance (up to 88%) was obtained across various stages of PD severity, as expressed by the Hoehn and Yahr (H&Y) scale. These values highlight the potential of our model as a non-invasive, cost-effective instrument for clinicians to remotely and accurately detect patients’ response to treatment from early to more advanced PD stages.