National Institutes of Health Stroke Scale scores obtained using a mobile application compared to the conventional paper form: a randomised controlled validation study

Background Prehospital delay contributes to treatment delay in acute stroke. Numerous prehospital stroke scales exist for stroke identification, but they lack the diagnostic accuracy of the in-hospital National Institutes of Health Stroke Scale (NIHSS). We have developed a mobile application to aid paramedics assessing prehospital NIHSS. This study explores agreement between NIHSS scores obtained using the Paramedic Norwegian Acute Stroke Prehospital Project (ParaNASPP) application compared with conventional assessment. Methods 25 physicians working with stroke were randomised to an application group or control. 20 unique videos portraying acute stroke symptoms were scored by both groups. 95% limits of agreement (LoA) were calculated using Bland-Altman’s method, comparing the groups to predefined scores, and each other. LoAs within ±3 NIHSS points were considered acceptable. Cohen’s kappa was calculated on dichotomised NIHSS scores. Results The ParaNASPP application group had 95% LoA of −2.33 to 2.71. The control group had LoA of −2.60 to 2.55. Direct comparison between the groups gave LoA of −3.12 to 3.55. When compared with the dichotomised predefined scores kappa was 0.93 in the application group and 0.89 in the control group. Kappa was 0.84 for direct comparison between the groups. Discussion There was very good agreement between the application and both the predefined score and the control group. Scores from the ParaNASPP application differ slightly more than our predefined goal when compared with the control group, but is well within when compared with the predefined NIHSS scores. We consider this acceptable and the ParaNASPP application validated for further clinical studies.