Rating inter-rater reliability of Shih–Hsu Test of attention between an experienced psychiatric occupational therapist and an occupational therapy student: A pilot study

Abstract

Objective: The “Shih-Hsu Test of Attention” (SHTA) is an iPad-based attention assessment tool developed by occupational therapists in recent years, and has acceptable criterion-related validity and high test–retest reliability in preliminary application. In this study, we intended to explore the inter-rater reliability of SHTA between experienced and inexperienced occupational therapists. Methods: We recruited 24 voluntary study participants aged 20–24 years in this study. The participants completed twice the SHTA by an experienced occupational therapist and an occupational therapy student. Results: Analytical results showed that the inter-rater reliability between experienced and inexperienced occupational therapists using SHTA had satisfactory reliability (intraclass correlation coefficient = 0.65). Conclusion: Our preliminary findings showed that the new attention assessment tool, SHTA, had satisfactory inter-rater reliability between experienced and occupational therapy students. We need to wait for a future study with more numbers of study participants in both groups to strengthen the study finding of this pilot study. At this moment, we suggest improving the guidance and training for inexperienced occupational therapists to improve accuracy, and reducing the gap when testing with experienced occupational therapists in future.